Complexity and Social Networks Blog

« April 2006 | Main | June 2006 »

29 May 2006

Looking for an answer? Get a little help from your friends' computers

Today's New York Times announces a new service from Tacit Software that will allow users to search for answers to their questions on the local computers of friends and colleagues - anonymously. The software (called Illumio) relies on a reverse auction model to determine who the "best expert" is in one's network. Sounds like a good idea to me, especially considering the problems people incur when searching for answers on listservs or through personal contacts. In a project I'm working on with David Lazer and Ines Mergel, we look at how people search for the knowledge they need to get their job done. Not surprisingly, we find that they generally don't look in the best places, but turn to those sources they are most comfortable with. We're hypothesizing that their comfort zone depends on their personality (a shy person would rather not post a question to a listserv, risking public humiliation in case their question is considered common knowledge) and the degree of familiarity with their peers, among others. On the response side, one's closest friends or those posting answers to questions on listservs might not be the "best experts" on a topic. (David and I will be presenting some early results of this project at the Academy of Management meeting in Atlanta; see our paper.)

Below is the full article from the Times.

May 29, 2006
Software to Look for Experts Among Your Friends
By JOHN MARKOFF

PALO ALTO, Calif., May 27 — For anyone who has hesitated before making a purchase on a Web site, uncertain which brand is preferable, Tacit Software is preparing to introduce an online service that will make it simple to pick the brains of friends and colleagues for opinions and expertise.

Tacit plans to start testing the service, called Illumio, next month. The service allows the user to mine the data on the computers of friends, business associates and others with shared interests on any subjects.

However, Illumio is not a search engine, like Google or Yahoo. The system works by transparently distributing a request for information on questions like "Who knows John Smith?" and "Are Nikon digital cameras better than Olympus?" to the computers in a network of users. The questions can then be answered locally based on a novel reverse auction system that Illumio uses to determine who the experts are.

The system is intended to extend a growing category of software that helps groups collaborate and work together more efficiently. Efforts to create systems that augment the intellectual power of work groups go back to the earliest days of computing technology development. The widespread availability of networks and Web browsers, however, has made such technologies far more accessible in recent years.

"The collaboration space is big and busy," said David L. Gilmour, president and chief executive of Tacit. "We don't consider ourselves a collaboration environment, rather we are about communication and search."

Currently, the privately held Tacit, which was founded in 1997, sells similar technology, known as ActiveNet, to corporate customers like Lockheed Martin, Northrop Grumman, Morgan Stanley and Sanofi-Aventis. The new Illumio version is intended to be used over the open Internet. It will be free for individual users and sold commercially to private groups, although the company has not announced pricing.

Software such as Illumio is representative of the rapid emergence of new markets for digital information, said Michael Schrage, a researcher at the Massachusetts Institute of Technology Sloan School of Management.

"This represents the eBayification of organizations," he said. "The reality is that organizations are run off of informal connections and tools such as this facilitate gray markets in information and interpersonal exchange."

Tacit's top achievement in its software for connecting people and expertise may be in a design that keeps personal information private.

"The biggest problem we had to solve was the privacy problem," Mr. Gilmour said.

Because the information used to determine if someone is an expert on a particular question stays on local computers, Tacit's executives said Illumio would avoid potentially troubling privacy questions. The Illumio software is installed on users' PC's, where it is connected through a software interface to either Microsoft or Google's desktop search programs that index local user content, including documents and electronic mail.

The anonymity offered by Illumio is a significant advantage over other social networking software services that place pressure on users to offer assistance.

The Illumio software uses a reverse auction model to restrict the answer to the best expert. In a reverse auction, sellers compete for the right to provide goods or services. For example, in response to the question, "Who knows John Smith?" each Illumio local system would independently determine who had the best relationship in the network based on parameters such as who had recently exchanged the most e-mail with John Smith.

If the local system found a strong relationship, the local Illumio client software would pop up a request on that user's screen asking whether the user wished to respond to the person asking the question. Initially only the strongest candidates would be notified locally of the query. If that user ignored the request, the reverse auction system would, in effect, lower the bar to ask the person with the next strongest relationship. Then, if there were no responses, the bar would be again lowered until an expert responded. It is possible that difficult questions would find no experts.

The system insures that experts remain anonymous until they agree to answer the query. When a user answers, the connection is made either through the Illumio system, by e-mail or by other channels such as instant messaging or telephone.

In addition to the keywords that make up the question, a user is permitted to send an accompanying message that will help people determine whether they have relevant information to a particular question.

Tacit hopes to market the service by providing Web masters with icons it calls "hot spots." For example, a person running a digital photography or similar Web site could place a hot spot on its home page and then anybody who wanted to join an Illumio network on digital photography could do so by simply clicking on the link.

If they already had the Illumio software, it would automatically add them to the group. If not, it would download and install both the indexing software and the Illumio client software. Illumio is currently available for Windows-based computers.

The potential of Illumio lies in its ability to help small groups of friends and associates tap expertise that they might otherwise not know existed, said Esther Dyson, publisher of Release 1.0, a computer industry newsletter and an Illumio investor. "This is searching your friends' heads as reflected in what's on their computers," Ms. Dyson said.

Posted by Maria Binz-Scharf at 3:15 PM | Comments (0)

26 May 2006

NSA phone log analysis

Excerpts from USAToday article on May 22nd that give greater insight into the types of analyses that the NSA may be doing.
Pre-9/11 records help flag suspicious calling
Updated 5/22/2006 11:46 PM ET
By John Diamond and Leslie Cauley, USA TODAY

The intelligence officials offered new insight into one way the database of calls is used to track terrorism suspects.
The officials, two current U.S. intelligence officials familiar with the program and two former U.S. intelligence officials, agreed to talk on condition of anonymity. The White House and the NSA refused to discuss the template or the program.

Using computer programs, the NSA searches through the database looking for suspicious calling patterns, the officials say. Because of the size of the database, virtually all the analysis is done by computer.
Calls coming into the country from Pakistan, Afghanistan or the Middle East, for example, are flagged by NSA computers if they are followed by a flood of calls from the number that received the call to other U.S. numbers.
The spy agency then checks the numbers against databases of phone numbers linked to terrorism, the officials say. Those include numbers found during searches of computers or cellphones that belonged to terrorists.

Posted by David Lazer at 11:03 AM | Comments (1)

18 May 2006

Citizen Relationship Management: the rocky road from transactions to empowernment

by Stephen F. King

The UK government has moved on from "electronic government" to "transformational government" (Cabinet Office, November 2005). "Customer-centric" public services and Customer Relationship Management (CRM) are seen as key elements of the new agenda. However, CRM originated in the private sector as a technology to support customer acquisition, retention and extension (cross-selling). The appropriateness of this technology to organisations striving to meet complex goals such as improving the quality of life for vulnerable people is open to question. Results of recent UK electronic government CRM programmes show that the focus for many local authorities has so far been systems integration, CRM-enabling call centres and the provision of routine transactions online. More advanced authorities are planning to use CRM to help them understand their citizens better. But more can be done. To this end, an alternative model of CRM progress is proposed which moves beyond transactions and customer insight and encourages citizens to co-produce the public services they consume.

An outline of this paper can be found here. The presentation is availabe on the Program on Networked Government Website. Stephen is looking forward to your comments.

Stephen F. King is a Senior Lecturer in Information Management at Leeds University Business School. He holds PhD in Information Systems from University of Warwick, an MPhil in Artificial Intelligence from University of Bradford and a BTech(Hons) in Computing from University of Bradford. He is a member of the British Computer Society and member of the UK Academy for Information Systems. Stephen King's research addresses five areas:
electronic government, electronic business, customer relationship management (CRM), enterprise resource planning (ERP) systems and electronic marketplaces.

Posted by Alexander Schellong at 9:34 AM | Comments (0)

11 May 2006

Science article: Finding Criminals Through DNA of Their Relatives

On a related note, regarding the data mining of relational data, today with co-authors Frederick Bieber and Charles Brenner, I had a paper come out in Science on using DNA databases to identify relatives of those in the databases: "Finding Criminals Through DNA of Their Relatives." In the extended entry is the Harvard press release.

In a small nutshell, we use a combination of Monte Carlo simulations and genetic kinship analysis to demonstrate the feasibility of identifying the source of a crime scene sample if a close relative of the source is in one of the offender DNA databases maintained by various jurisdictions around the country (and the world). We find, for example, that in a database of 50,000, if one rank ordered the most likely close relatives to least likely close relatives, if a close relative were in the database, that relative would be ranked first about half of the time, and in the top 100 99% or so of the time. Accuracy could be improved further if additional genetic data were extracted from the samples. In short, kinship analysis would indirectly incorporate most of the first degree relatives of those in DNA databases (there are 3+m people in US databases). We discuss the policy and ethical implications of these findings, which I will not go into here (see release below).

There are two interesting things, from the perspective of this bog, that I would like to note. First, is the paradigm shift that kinship analysis represents. If one looks at the discourse around the expansion of DNA databases in this country, it has focused on the attributes of those going into the database-- convicts for certain crimes, what is the probability of recidivism, etc. But genetic data is inherently relational, and thus the technology is really incompatible with the discourse. The interesting thing--although, perhaps this paper will just drop into the bottomless well, never to be heard of again-- is what happens when credible research comes out highlighting the disjuncture.

Second, is the more general issue around data mining of relational data, individual choice, and privacy. As both this article and the NSA data mining highlight, so much information is inherently relational. The fact that I have certain characteristics may say something about people that I have various types of connections to. This does create certain conundrums for constructions of privacy that rely on individual choice, since with relational information, what I choose to reveal about myself reveals something about others-- i.e., there are informational externalities. In this day and age, this issue is endemic, and suggests that certain types of decisions about privacy must be necessarily communal and not individual in nature.

Using Relatives' DNA to Identify Criminals
Potential problems and benefits identified in new research study

Genetic analysis of proliferating DNA databases maintained by the criminal justice system might enable criminals to be caught by recognizing, as their kin, those in these databases. While use of these novel methods could provide thousands of new investigative leads, they also raise challenging legal and policy questions. These findings along with discussion of the legal and policy issues are published in ScienceExpress, the advance online version of Science, on May 11, 2006.

The article is co-authored by Frederick R. Bieber, PhD, a medical geneticist at Brigham and Women’s Hospital and Associate Professor of Pathology at Harvard Medical School; Charles H. Brenner, PhD, a forensic mathematician and Visiting Scholar at the School of Public Health, University of California, Berkeley; and David Lazer, PhD, Director of the Program on Networked Governance and Associate Professor at Harvard’s Kennedy School of Government.

“We demonstrate that the kinship analysis methods routinely used for humanitarian mass disaster and missing persons identifications could also be used to identify criminals who are close relatives of those in the rapidly expanding offender DNA databases,” says Bieber.

Currently there are about 3 million individuals in offender databases around the country, largely including individuals convicted and arrested of serious crimes. The genetic profiles of these individuals are then compared to evidence of crime scene evidence, potentially linking known individuals to crime scenes. The team evaluated the potential for using these databases to link crimes to individuals related to those in the database by analyzing data on existing genetic markers used in forensics to calculate likelihood ratios (LRs) of potential parent/child and sibling relationships between each offender in the a simulated database and a crime scene DNA profile. They did this to assess the chance that a crime scene sample belongs to the close relative of a cataloged offender. The higher the LR is, the more likely that the crime scene DNA sample belongs to a close relative in the offender database. “The simulations illustrate that kinship searching used in conjunction with follow-up analysis, preferably using Y-chromosomal–markers, can expand the reach of the DNA databases not by handfuls, but by gross amounts,” said Brenner.

Researchers note two important caveats, however; that although unusual, a high LR may be a result of a coincidence for unrelated individuals and that a DNA match does not necessarily prove guilt.

“Kinship analysis, as with any investigative technique, may lead to the investigation of the innocent,” says Lazer. “But it would also provide decisive leads in cases that would otherwise go unsolved. Conceivably it could provide a large number of cold hits.” A cold hit refers to the chance of finding a match between a crime scene sample and someone in the offender database.

Kinship analysis would also reflect the current composition of the database. “The database is disproportionately African American, Hispanic, poor, and those who would be indirectly included through kinship analysis would reflect those disparities,” says Lazer.

Opposition to DNA databases has been in the form of protection against civil liberties, specifically privacy. However, courts thus far have ruled that public safety concerns outweigh individual privacy rights regarding offender DNA databases. Current laws, both state and federal, do not address kinship searches, probably because these strategies were not considered when the original laws were drafted.

“While it is exciting and important to acknowledge the potential value that genetic kinship analysis could have in forensic investigations, it is also necessary to take heed of the legal and policy implications,” says Bieber.

These and related issues will be discussed at a DNA Fingerprinting and Civil Liberties National Symposium being held in Boston, May 11-13, sponsored by the American Society of Law, Medicine and Ethics.

Posted by David Lazer at 10:04 PM | Comments (0)

NSA data collections

In my earlier postings on the NSA data gathering program, I posed the question, what data would I want if I were the NSA, and all I cared about was potentially detecting terrorist activity, etc. My answer was simply, I would want them all. Further, I pointed out that the principles laid out by the administration did little to point to any line short of "all" that was desirable. Why stop with international calls? etc.

This was closer to the truth than I imagined, per USAToday story that broke today. Excerpts:

"It's the largest database ever assembled in the world," said one person, who, like the others who agreed to talk about the NSA's activities, declined to be identified by name or affiliation. The agency's goal is "to create a database of every call ever made" within the nation's borders, this person added.
…
In defending the previously disclosed program, Bush insisted that the NSA was focused exclusively on international calls. "In other words," Bush explained, "one end of the communication must be outside the United States."…
Sources, however, say that is not the case. With access to records of billions of domestic calls, the NSA has gained a secret window into the communications habits of millions of Americans.
…

The data are used for "social network analysis," the official said, meaning to study how terrorist networks contact each other and how they are tied together.

Posted by David Lazer at 9:11 PM | Comments (1)

9 May 2006

social psych and social network analysis

Speaking of fields and social network analysis, psychology offers an interesting puzzle for the relative paucity of psych-related social network analysis. Much of the classic early work in network analysis involved psychologists: Newcomb, Bavelas, Festinger, Heider, the Robbers Cave experiments. Yet in the 1960s (my impression is) the field veered away from social network analysis, which took refuge in sociology and anthropology. Perhaps this is due to power of the Asch experiments, which paradigmatically displaced much of the other work (Newcomb, Festinger) on social influence, bringing the focus more to the group level than network.

Posted by David Lazer at 8:21 AM | Comments (1)

8 May 2006

Unknowability: Implications of Darwinian Preadaptions for Economic Growth and Policy

by Stuart A. Kauffman

Stunningly, Darwinian preadaptations imply that we cannot finitely prestate the evolution of the biosphere, nor can we prestate the future evoluition of technology, let alone history. In all cases, we literally do not know beforehand the relevant variables. The biosphere has dealt with this fundamental fact by the evolution of the very capacity to evolve and adapt. During this seminar, professor Kauffman will discuss these facts, and what we begin to know about the capacity to be able to adapt. The hints are that this capacity is maximized at a phase transition between order and chaos. This seminar will also attempt to extend these and recent work on agent based models to the arena of policy formulation.

Presentation documents will be available soon.

Stuart A. Kauffman is presently a professor at the University of Calgary with a joint appointment between biological sciences and physics and astronomy.

He is a theoretical biologist who studies the origin of life and the origins of molecular organization. He is a MacArthur Fellow and an external professor at the Santa Fe Institute. Thirty-five years ago, he developed the Kauffman models, which are random networks exhibiting a kind of self-organization that he terms "order for free." Dr. Kauffman was the founding general partner and chief scientific officer of The Bios Group, a company (acquired in 2003 by NuTech Solutions) that applies the science of complexity to business management problems. He is the author of The Origins of Order, Investigations, and At Home in the Universe: The Search for the Laws of Self-Organization.

Posted by Alexander Schellong at 11:01 AM | Comments (0)

5 May 2006

Putting the network into politics (or at least political science)

Despite my best efforts, there is relatively little use of network analysis within political science. This was highlighted by my experiences at the Sunbelt and the Midwest Political Science Association meetings (the second largest meetings in political science, I believe) the last few weeks. I think the overlap between the two conferences was… me. There is a small upsurge in network-related research in the field—I am guessing there were several times as many papers at MPSA using network ideas than in previous years. But I doubt that I would need more than two hands to count the number of network-related papers at MPSA, which is a pretty big conference.

This despite, I would argue, the obvious applicability of network ideas to political science. How natural to look at politics as a product of the network of connections among agents with differential resources, where both the resources and connections make a difference to outcomes of interest. One simply has to look at the money that lobbyists make to infer that social capital may be converted into political capital. However, there is a rather meager amount of research in this vein within political science—instead, such analyses have appeared in sociology (e.g., Laumann and Knoke’s The Organizational State). There are some exceptions—e.g., the papers I wrote with Dan Carpenter and Kevin Esterling using the Laumann-Knoke data; Young’s book on the social organization of Congress as well as Patterson’s work on the personal networks of state legislatures (also see recent paper by Jim Fowler looking at co-sponsorship networks in Congress). There has also been some efforts bringing network ideas into the flow of things through the international system—including my own work, as well as Fabrizio Gilardi and others. Most prominent within the field has been a small but important vein of research by Robert Huckfeldt, John Sprague, and colleagues in the spirit of the Columbia studies on social networks and political opinion. Interestingly (this is just an impression), the social capital boom, instigated in significant part by my colleague Robert Putnam—a political scientist— has been far less in political science than in other social science fields, e.g., sociology, organizational behavior, and (surprisingly) now in economics. I think, probably, now, within political science, the field within which network ideas are most in vogue is public administration, where scholars such as Larry O’Toole, Brint Milward, Ken Meier, John Scholz and others, have been writing about the inter-linking of government and non-government actors for a while.

[If people can think of other examples, please post.]

Why this lack of network related research? I don’t have a definitive answer; and because of the feudal nature of political science, one has to do a fiefdom by fiefdom analysis. In part, it reflects (within American politics) the organization of the field in significant part around institutions—Congress, the Presidency, etc. Further, when one looks at actors, it is typically assumed that they are purposive/rational and independent. Public opinion research (with the exception of Huckfeldt, etc) relies on the essential independence of observations—an assumption that social influence, etc, peels away. International relations has focused in part on the (neo)realist vs (neo) liberal debate, which reifies states and again often assumes away any particular structure connecting them. There have been studies (e.g., on the “democratic peace�?) which have done dyadic analyses of the predictors of conflict—however, these studies (the last I looked at them) assumed an independence of dyads—whether A and B are at conflict is essentially independent of whether B and C are at conflict. This is an assumption that is clearly at odds with casual observation and something amenable to analysis with modern network-based statistical methods (e.g., p*).

So, looking for a generalization across fields, there seem to be paradigmatic foci at the institutional level and actor levels (usually utilizing assumptions of rationality regarding the latter), but a relative paucity of theorizing at levels between—which is where network analysis exists.

Posted by David Lazer at 8:16 AM | Comments (3)

3 May 2006

Sunbelt, evolving

The first Sunbelt I attended was in 2000 (also in Vancouver). It is striking how the conference has evolved in just the last 6 years—obviously, reflecting the increased interest in the field. It is much larger than it was then (maybe 50% ??). Some of this growth may be ephemeral—about half the attendees, I think, were fist timers, while I am betting that maybe 10% had attended going back to the 1980s. The composition of fields represented has also evolved. While always interdisciplinary, my perception is that there has been a shift from sociology to public health, organizational behavior, and information and computer scientists. There is also significant private sector interest, with folks from, for example, Microsoft and Yahoo, among many others. There was, I understand, also representation from government (rather less visible). There were few anthropologists, prominent in earlier years, and (alas), other than me, few political scientists, which reflects the still low levels of interest in political science about networks. (In my next posting I will discuss why there has been little social network analysis within political science, as well as the stirrings of interest in recent years.) Some other impressions: as noted in the previous post, there has been an increase in use of behavioral data. There were definitely also more simulation models than in previous years. And there has definitely been an increase in the study (and tools for analysis) of longitudinal data. For any readers who attended Sunbelt 2006 and earlier editions, I would be interested in your observations.

Posted by David Lazer at 8:26 AM | Comments (0)

1 May 2006

Sunbelt 2006

I write this as I return from the 2006 edition of the Sunbelt (social network) conference. I have had the privilege of many unanticipated bonus hours in various airports around the continent that have allowed me to get work done. I did not have time to post anything while I was in Vancouver, but my next few posts will be on Sunbelt, as well as on the political science conference I attended the previous week (the Midwest—it has been a busy couple of weeks). Today I will write about some of the observations from Sunbelt; later this week I want to discuss the contrast between these two conferences.

One of the more interesting presentations I attended was by Jim Moody of Duke. He presented some of the work he has been doing on dynamic networks, showing some of the moving pictures of networks changing over time. What was particularly striking to me was his illustration of the principle that when one takes into account timing of the flow of things through a network, ones perspective on key structural features of the network fundamentally changes. For example, the most central node in a network, e.g., as measured by betweenness, can look peripheral if one takes into account the timing of flow. If the most central node gets information slowly, information will flow around that node. A dynamic picture of flow in a network thus can produce a fundamentally different understanding of the structure of the network than a static picture. (I should note that a dynamic picture does not mean necessarily that the network is changing—just that there may be a natural sequence in communication, which may be a long standing structural feature of the network. That is, there is a difference between saying that networks are dynamic and that networks evolve over time.)

This isn’t a totally novel idea—in fact, I think it has been explored in the various research on traffic networks (a field that predates social network analysis), which has always dealt with networks and flow (Ana Nagurney did a very nice job of providing an overview of this field at her talk in the Fall in CCCSN). Nonetheless, it was something that I had not totally appreciated before.

This links back to something that I have written about earlier on the blog—the coming revolution in the study of social networks using behavioral data. It will, in fact, be possible to do an analysis of the scheduling of flows over networks in a fashion heretofore impossible (e.g., to see that Joe talks to Ralph, who then talks to Anne, etc).

A final note on the coming behavioral revolution—I made a wager (dinner) with David Krackhardt that 10 years from now the majority of presentations in Sunbelt will be using behavioral data—e-mail, phone, log, blog link structure, etc. There were certainly a lot more of these kinds of presentations in 2006 than in past years.

Posted by David Lazer at 11:01 AM | Comments (0)