Complexity and Social Networks Blog

« November 2005 | Main | January 2006 »

30 December 2005

Social network analysis, the NSA, and “pattern analysis�

The story about the NSA eavesdropping program has received a lot of attention over the last week. The follow up story has received somewhat less attention, but may be more important, see story from December 24 NYT: Spy Agency Mined Vast Data Trove, Officials Report (by ERIC LICHTBLAU and JAMES RISEN):

“What has not been publicly acknowledged is that N.S.A. technicians, besides actually eavesdropping on specific conversations, have combed through large volumes of phone and Internet traffic in search of patterns that might point to terrorism suspects. Some officials describe the program as a large data-mining operation.�

“A former technology manager at a major telecommunications company said that since the Sept. 11 attacks, the leading companies in the industry have been storing information on calling patterns and giving it to the federal government to aid in tracking possible terrorists.

"All that data is mined with the cooperation of the government and shared with them, and since 9/11, there's been much more active involvement in that area," said the former manager…

This is a remarkable story, and raises some interesting questions: (1) exactly what data are telecomm companies sharing with the government; (2) what could usefully be gleaned from these data; and (3) what are the privacy implications?

There is a lot more we don’t know about this story than we do know, but it is worth beginning a discussion on the value and the costs of these data under different scenarios of exactly what information is being shared. My next few entries will aim to begin a discussion on these issues, grounded primarily in a social network perspective.

Briefly, what data do telecomm companies have? Focusing on the telephone data, for now, they have (1) phone log data; (2) varying amounts of locational information for cell phones; and (3) varying amounts of information linking individuals to particular phone numbers (e.g., not so much for some pay as you go phones, more for other types of phones). My understanding is that little remains of the bits that flow over the network (i.e., the content).

These are thus a type of social network data, along the lines of my preceding entry on the “behavioral flows� of relationships. That is, for any given dyad one can observe the timing and duration of calls.

Whose phone data is being tracked? It is not clear from the article. Clearly, the focus is on international communication (domestic to international, and international to international calls routed through switches that are on US soil). Is purely domestic communication also being tracked? The article suggests not:

“This so-called "pattern analysis" on calls within the United States would, in many circumstances, require a court warrant if the government wanted to trace who calls whom.�

This sentence is ambiguous, however—e.g., given that the sharing of data by the telecomm companies is voluntary, what are the statutory limits on their sharing data with the government? Is there a prohibition on a telecomm company voluntarily handing over information to the government regarding one of their customers’ phone logs? I do not know of such a prohibition, but if a reader does, please do comment.

For the next entry: Given that these are essentially social network data, from what we know from the research on social networks, what insights might they yield?

Posted by David Lazer at 11:19 PM | Comments (3)

22 December 2005

Virtual Stock Markets - Proving the Powerlaw?

Social relations between individuals can be complex systems. How the structure of social networks impacts the behaviour of a system has been researched recently. These are i.e. power grids, neural networks, the World Wide Web or stock markets. Although different in the underlying interaction dynamics or micro-physics, all these networks have shown a tendency to self-organize in structures that share common features. In particular, the number of connections, for each element, or node, of the network follow a power law distribution. Networks that fulfill this property are referred to as scale-free (SF) networks M. Bartolozzi, D. B. Leinweber1, A. W. Thomas. (2005).

I would like to draw your attention to 2 projects which are using the power law in a direct and indirect way. First, there is the use of virtual stock markets to improve market research. Second, a recent project concerning blogs and virtual stock markets (VSMs) tries to proove the existence of powerlaw.

VSMs aven been applied in the form of a political stock market to predict the outcome of US presidential election in 1988 at the University of Iowa. The results of these studies demonstrate that the predictions outperform opinion polls in terms of forecast accuracy. Furthermore, the results of political stock markets show that VSMs can perform well even if their participants are not a representative sample. The reason is that VSMs elicit the participants' assessments of the market outcome and a rational participant should not trade according to individual preferences, but according to the prediction of the market outcome based on the overall preferences of market participants. Thus, the decision is based on the most common features an individual anticipates in the market (powerlaw). More virtual stock market research is in this area is currently underway by an international research team (Martin Spann (U Passau), Gerrit van Bruggen (EU Rotterdam), Ely Dahan (UCLA) and Gary Lilien (U Penn)). Although it is more focused on business and market research some outcomes might be useful in other research areas.

BlogShares is the exploration of an emerging social network. Blogs are valued by their incoming links from other blogs. A blog is defined as a company and links become the main source of value in the VSM. Players speculate on thousands of blogs by buying and selling shares or rather the shifts of attention within the network. Blogshares claims to have proven the powerlaw which is in this case that 20% of the blogs contain 80% of all incoming links.

Posted by Alexander Schellong at 5:48 PM | Comments (2)

21 December 2005

Auld lang syne: networks as behavioral flows

Over this holiday season, as many of you go somewhere, or are the destination for others, I hope you will be thinking about social networks. In particular, I hope you will think about social networks using a different metaphor than is usually applied in the field. The dominant way of thinking about networks, still, is as a slow-changing structure that enables/constrains behavior, and/or through which things circulate. While I think that this metaphor has value, this season think about “the network� as a set of relational behavioral flows, where you engage in some set of behaviors (visiting, writing Christmas cards, e-mailing, etc) vis-à-vis other people. A relationship is thus simply a particular behavior at a particular moment in time, and networks are simply the accumulation of these moments over time for some set of people. Networks, thus viewed, may exhibit certain types of properties, e.g., periodicity. Nathan Eagle of the Media Lab (with Sandy Pentland), has done some particularly nifty work with devices that measure interactions (“sociometers�), demonstrating the kinds of periodicity one may see among people who work together. Holiday travel is another example, where certain pairs (and larger order aggregations) may tend to get together at a particular time of year over an extended period. Further, certain types of events (e.g., graduating, getting a new job, etc), through this lens, are simply correlates of dramatic changes in these behavioral flows.

This metaphor, I think, can take you some places that a structural metaphor cannot—e.g., in understanding the spread of things through the system. Further, in turn, it can strengthen the structural metaphor by understanding the micro-behavioral foundations of certain relationships. E.g., Eagle finds that friends have systematically different relational flows than non-friends. The relational flow metaphor also highlights sequence in a way that is invisible in the structural metaphor. This is something that David Gibson (formerly of Harvard, now at Penn) has done some nice work on (Mazel tov to David and Ann on the new addition to their family, btw!).

Just something to think about as you are sitting down for your holiday meal, and singing auld lang syne (which itself is about the ebb and flow of relationships).

Refs:

Gibson, David R. 2005. "Concurrency and Commitment: Network Scheduling and its Consequences for Diffusion." Journal of Mathematical Sociology 29:295-323

http://www.nathaneagle.com/

Posted by David Lazer at 7:54 AM | Comments (4)

19 December 2005

Causal consulting

My sense is that social network analysis has increasingly been used for consulting purposes. This raises a couple of concerns and an opportunity. The concerns are two-fold: first is that a body of complex and sometimes conflicting findings are inevitably hyped and simplified as they pass through the prism of the consulting world—I think sometimes beyond recognition. Second is that, as noted in my previous posts, a lot of these findings rest on fairly shaky causal legs—particularly when you consider the lack of studies on system-level network structure and system performance. That is, perhaps importing these ideas into practice is the organizational equivalent of hormone replacement therapy. We make prescriptions based on correlational evidence, and make recommendations that may have adverse effects.

That said, ultimately ideas only matter if they have some impact on how people think and act—that is, people outside of the insular world of academia. One hopes that SNA can offer insights into how organizations (and other collectives) function, and how to operate more effectively. This all points back to my earlier arguments about the need to strengthen the foundations of causal assertions in the field.

This, in turn, points back to what (consultant and other based) interventions can offer back to the field—better insight into cause and effect. For example, do particular types of “network strengthening� actually improve outcomes at group and individual levels as predicted? Does making expertise and social networks transparent increase knowledge sharing? Do efforts to increase relationships across silo’s improve coordination and access to information? And are there any unanticipated negative consequences? Etc etc. Of course, all of this presupposes building in evaluative measures into the intervention, and then a rigorous evaluation of whether the intervention worked, and it may not be reasonable to expect those that recommend certain interventions to rigorously evaluate them. But one problem at a time….

Posted by David Lazer at 10:51 PM | Comments (4)

12 December 2005

Talk by Anna Nagurney

Anna Nagurney
University of Massachusetts, Amherst
and
Radcliffe Institute Fellow, Harvard University

The Evolution and Integration of Social and Financial Networks with Applications (PDF)

Monday, December 12, 2005, 12:00 - 1:30 p.m.
Bell Hall, John F. Kennedy School of Government

This seminar is co-sponsored by the Institute for Quantitative Social Science

In this talk we will overview some of the methodological tools that we are using and developing in order to model the integration of social networks with economic networks, notably, supply chain and financial networks.

Anna Nagurney is the John F. Smith Memorial Professor in the Department of Finance and Operations Management in the Isenberg School of Management at the University of Massachusetts at Amherst. She is also the Founding Director of the Virtual Center for Supernetworks and the Supernetworks Laboratory for Computation and Visualization at UMass Amherst. She received her AB, ScB, ScM, and PhD degrees from Brown University in Providence, Rhode Island. She devotes her career to education and research that combines management, economics, and engineering. Her focus is the applied and theoretical aspects of network systems, particularly in the areas of transportation and logistics and economics and finance. She is the editor of the book, Innovations in Financial and Economic Networks (November 2003), and has authored or co-authored 8 other books including Supernetworks: Decision-Making for the Information Age, Financial Networks, Sustainable Transportation Networks, and Network Economics.

Posted by David Lazer at 11:38 AM | Comments (5)

7 December 2005

An Introduction

As a way of introducing myself to this blog, I'm posting an interview that I recently did for a radio show on the Canadian Broadcasting Corporation (CBC). This interview is part of a series about fellows at Massey College, University of Toronto. The interview focuses on my research – how people maintain their relationships by way of the internet. I hope you enjoy. To Listen, Click Here.

Posted by Jeff Boase at 10:55 PM | Comments (0)

6 December 2005

Christmas cards and network pings

Thomas Schelling, in Micromotives and Macrobehavior, notes the social inefficiencies of Christmas cards (31-33). He claims that cards are based on a system of obligations, and that once a card has been sent one year, failing to send one each subsequent years sends and active—and negative—signal. Tim Hartford, the Undercover Economist, reiterates this theme, and notes that the yearly lag for feedback (I send a card this year, and receive one in return next year) may create this trap. Instead of thinking of the card tradition as a forced system of obligations, however, suppose we look at it from a networks perspective.

Routers on the internet occasionally send “pings� to other machines, to maintain a set of contacts and reaffirm each other’s existence. A ping packet is very simple, and the transaction carries little contextual information beyond the necessities of network config data. I would argue that cards serve a similar function: the simplest form of communication between two parties to maintain a network of weak ties. There are many people with whom I would like to maintain some casual ties, but have relatively little to say at this time. An informal poll of colleagues in the office supported this view: people want to maintain networks beyond the daily standards of intimacy. The basic problem with cards is that we now have simpler and cheaper forms of communication, so the relative cost of card + labor + stamp has grown.

If we were to grant to traditionalists that the traditional practice of pen-and-paper correspondence has largely disappeared from modern life, there are still ample ways to stay in touch with friends and colleagues. Phone, email and instant message, for example, are now relatively cheap and ubiquitous in certain networks, yet each medium carries a certain overhead in the missive. A phone conversation, particularly without a specific motive, can be awkward and requires small talk “catch up� that may not be necessary or wanted. The synchronous nature of phone calls make them hard to scale for a large ego-network, especially given time constraints. IM can interrupt the alter at his or her workspace, which can be equally invasive, and is also subject to time constraints. IM does allow a constant presence through the practice of “buddy lists� allowing presence to be registered without active communication. (Has anyone done research on presence and maintenance of buddy lists?) However, this practice may be too passive to “ping� a network tie: like a blog or website, it is pull, rather than push.

So is it possible to have a social network ping mechanism that is low cost but socially acceptable? Or is the inherent cost of the ping integral to maintenance of social ties, a la economics of gift giving? Hartford argues that we should burn the Christmas card list and start over again with people we actually care for. Perhaps a push to move the annual holiday greetings to an email format would be better. Go ahead, ping your network.

Posted by Allan Friedman at 11:18 AM | Comments (0)

5 December 2005

Longitudinal data, causal inferences, and the institutional milieu

A quick follow up on my earlier post re causal inferences, in which I stated that longitudinal data are not a cure all for determining the direction of the causal arrow. Longitudinal data, in principle, should allow a tracing of what preceded what temporally, and thus (hopefully) causally. Thus, in the context of social influence, if A and B started talking at time t, and their attitudes converged at time t + 1, it would seem reasonable to assert that communication lead to convergence (social influence), rather than prior similarity to communication (homophily). However, it possible that exogenous factors are dynamically operating on either the network or attitudes (to take the social influence example) over time. For example, imagine the attitude in question has to do with the role of government in markets, and one found looking at a population that both attitudes and communication patterns converged over time (suggesting both homophily and social influence). But now add to this scenario that the population in question is an undergraduate cohort, and an alternative explanation might simply be that ones major (e.g., economics) affects both attitudes and ties over time. It is plausible that such a process would lead to incorrect inferences regarding the sources of attitudes and ties. More generally, institutions affect both outcomes of interest and the configuration of networks. Neglect of the institutional milieu (which is often ignored in SNA research) can thus lead to spurious inferences regarding the reciprocal influence of networks and individual-level outcomes, even with longitudinal data.

Posted by David Lazer at 1:58 PM | Comments (0)

1 December 2005

Control and causation

Following up on both the Mobius paper and my earlier passage on causation, how many social network related studies have incorporated a degree of control over some critical dimension of the network and/or other factors? In the Mobius paper resources, of a sort, were exogenously spread through the network, with certain paths facilitated (through lower interest rates for certain dyads) and their diffusion studied. In Festinger’s classic study on social influence, people were exogenously placed in housing. In Newcomb’s dorm study the students were exogenously placed in their rooms, and various measurements taken before they took residence. There are also small group lab experiments—Bavelas and colleagues’ work in the Small Group Network Laboratory at MIT in the 1950s—and social exchange theory—by Emerson and Cook and others in the 1970s and after. What other research has there been where there has been a degree of control? I interpret control pretty liberally here—e.g., where some exogenous proxy, correlated with communication, is used to examine the impact of the network. It’s an intrinsically difficult problem, since typically one cannot randomly assign certain types of relationships to pairs of people ("you two be friends"), but not insurmountable. If people have (1) ideas re particular research that did have a measure of control; or (2) ideas re how to achieve a degree of control so as to allow better causal inferences, please post a comment.

Posted by David Lazer at 10:08 AM | Comments (1)