Main

May 1, 2009

CDC is fighting the spread of the swine flu with viral technologies

The CDC (Center for Disease Control and Prevention) is using several different social media channels to inform about the swine flu besides the traditional (Web 1.0) channels, such as frequent press briefings, general information in audio and video, etc.:


  1. Updates from the H1N1 page haven an RSS feed.

  2. Frequent updates are spread using Twitter.

  3. Video updates are posted using podcasts.

  4. Image sharing on the CDCs Flickr site.

  5. Buttons for your website.

  6. Information sharing on MySpace's e-health page and daily strength group.

  7. Updates can be shared using several different services (Google Reader, Bookmarks, Delicious, Facebook, Digg, etc.).

  8. e-Cards to send by email to family members and friends to remind people to wash their hands.

  9. Agencies can embed a flu widget on their page.

On the funny side: Do you have swine flu?

April 13, 2009

Twitter? You can not be serious!!!!

Do any of you actually use Twitter for work?

twitter.png


Sure, it's fun, and does something that Facebook does a bit better ("What's on your mind?").
I see it as the next generation of social software ......
friendster --> myspace --> facebook --> twitter
(with of course hundreds of other, equivalent, competing platforms)

But to use it for work stuff? To use it for anything more than goofing around with your buddies? Really? You are not serious, are you?


February 22, 2009

Facebook's Terms of Use and implications for network researchers

The changes of Facebook's Terms of Use were quickly followed by massive protests of thousands of users requesting to abandon those changes. The Consumerist Blog was one of the first to ask their readers to boycott Facebook and look for alternative ways to connect with friends.

About a week after the change, Facebook made the decision to revert back to their original TOS (from Sepember 2008) and now works with their lawyers and legal specialists to come up with an improved version.

For researchers the TOS are critical: not just for understanding how Facebook will use our own data, but we also need to understand how we can use network data to analyze emergent social structures and the way users create, maintain, or abandon their online ties. The current TOS leave us in limbo - not knowing what is allowed and to what extent.

To understand this better and to collect the wisdom of the social network analyst crowd, I recently started a discussion on this topic on the SocNet listserver. I am trying to find arguments that will help to explain my research interests to an Institutional Review Board. The discussion is still going on. A few highlights are:

  • Facebook does not allow research (or anyone) to store data more than 24 hours, which makes it difficult to clean, analyze and of course at the end publish the data
  • Data needs to be anonymous (especially in SNA network data cannot be anonymous - we need to know what kind of actors are nominating other actors and longitudinal data analysis seems to be impossible)
  • So far I have identified three different ways to collect/use Facebook data, although at this point it is unclear how people can comply to the first two bullet points.
1. Bernie Hogan at the Oxford Internet Institute, University of Oxford, UK, has created a Facebook application available on iTunesU to analyze Facebook data (open iTunes -> iTunes -> Oxford University).

2. Dataverse project at Harvard's Berkman Center has made available Facebook data.

3. Create an application or a group on Facebook where you can find a way to have people give their consent to collect data on their online behavior and contacts.

We have set up an informal meeting at the annual INSNA (International Network of Social Network Analysis) conference in San Diego to exchange some of the ideas and information available. In case you are interested in joining us - please email me at ines_mergel(at)yahoo.com. I will post an update after the conference in March.

February 3, 2009

Christakis and Fowler in SCIENCE, 23 Jan 2009 Issue

The golden boys of networks in public health are featured in this week's issue of SCIENCE.christakis fowler photo.png

From the headline of the article:

Friendship as a Health Factor
In a string of hot articles, two social scientists report that obesity, smoking, and other facets of health "spread" in networks. As the two friends expand their theory, doubters sharpen their questions.

The story is way cool. Their research, and the ground that they are breaking, are way cooler.

November 24, 2008

Social Media vs. Knowledge Management

Just a short post. Stumbled on an interesting article in Social Computing mag.

October 26, 2008

Few Secrets on the N.F.L. Grapevine

If you are looking for a colorful example of "social capital" check out article in sports section of today's New York Times, examining the flow of "intelligence" among NFL teams. It is an interesting study in the ethics of sharing insider information.

Excerpts below:

Spreading the Word Is No Secret in the N.F.L.

"Every team that you play against, your friends who play for other teams are like: 'Hey, tell me about this. Tell me about that. This defense, how does it run its scheme? Do they tip it?' " Washington Redskins running back Shaun Alexander said. "That's what happens. It's like advance scouting. You do the best scouting through your friends."

...

One look at the playing field in the hours before kickoff -- when players and coaches from both teams mingle as they warm up -- reveals the close ties that form the underpinnings of the N.F.L. Players who went to high school or college together catch up, although some do that the night before the game, when they may go out to dinner together. Coaches who may have been colleagues on the same staff early in their careers trade war stories. Even scouts, seeking confirmation of their player evaluations, check in with their peers as preparations for the draft begin.

"I probably knew coaches and scouts on every one of the 32 teams," said Steve Mariucci, the former San Francisco and Detroit coach who is now an analyst for the NFL Network. "You can't keep your head in the sand and assume your team knows everything there is to know. Communicating with your friends is not only accepted, it's necessary. Asking a question -- what did you think about so-and-so, what about their plan against you? -- those conversations can occur. No harm, no foul. It's commonplace."

But what makes the sharing of information all but inevitable is how often players and coaches switch teams. Coaches frequently call former colleagues during the season, and if their old friend just happened to have faced a future foe, so much the better. And if a friend coaches a team that is about to play a division rival, he will pour out the details.

...

One broadcaster said he was told that before the Baltimore Ravens played the Cincinnati Bengals in their season opener, the Ravens asked offensive lineman Willie Anderson -- who signed with Baltimore days after the Bengals cut him at the end of the preseason after 12 seasons -- why the Bengals had so much success blocking the Ravens' blitz. Anderson gave the Ravens' defense the Bengals' offensive line calls. The Bengals had not changed them. The Ravens were so effective that Bengals receiver Chad Ocho Cinco actually praised the blitz-oriented defense after the game.

"Is Baltimore the bad guy?" the broadcaster said rhetorically. "Is Willie Anderson the bad guy?"

October 19, 2008

Pew study: the connected American family

A just released Pew report, Networked Families (by Tracy Kennedy, Aaron Smith, Amy Tracy Wells, and Barry Wellman), may be of interest to some readers of the blog. To quote from the abstract:

The internet and cell phones have become central components of modern family life. Among all household types, the traditional nuclear family has the highest rate of technology usage and ownership.

A national survey has found that households with a married couple and minor children are more likely than other household types -- such as single adults, homes with unrelated adults, or couples without children to have cell phones and use the internet.

The survey shows that these high rates of technology ownership affect family life. In particular, cell phones allow family members to stay more regularly in touch even when they are not physically together. Moreover, many members of married-with-children households view material online together.

Also, go to the Washington Post story.

The report is chock full of interesting data on how modern communication technologies have become integrated into the daily lives of American families. I am a bit concerned, however, about the interpretation that these technologies have actually improved communication within families. For example, respondents are asked whether the Internet has improved connections to family members/friends/etc, and given the choices a lot, some, only a little, not at all. It is problematic to interpret these particular results as supporting the proposition that the Internet has improved communication within families, because (1) no choices are given that the Internet has undermined connections, and, more important, (2) respondents are not in a position to construct a counterfactual. (At least, I can no more imagine what communication within my family would be like without the Internet than it would be without the automobile.) That is, I would be cautious in interpreting some of these results as supporting the proposition that the Internet, for example, really has improved communication within the family.

Interesting puzzle to me, in a question to adults (from p. 28) about whether the Internet and cell phones have made their families closer than their families when they grew up, 28% of families with cell phones and Internet in their households indicate that these technologies have made their families closer, but, oddly, 17% of families without either also indicate that these technologies have made their families closer. It is striking to me that there is not a bigger difference-- is this because of indirect access to these technologies (at school, work, through friends)?

Neat factoid from report (p. 24): women generally seem to communicate more with their children than men, but there is a particularly large gender gap in text messaging with children, with 28% of women reporting texting their kids, and only 12% of men. I am also surprised that as many as one in twenty five parents communicate with their children (age 7-17, so this includes many parents who don't have kids who are using these Facebook, etc) via social networking sites at least once a day. My oldest considers it scandalous that I am even on Facebook, and has flatly forbidden me from friending her....

September 14, 2008

Obama's Neighbor to Neighbor tool

I have written before about the network-based foundation of Obama's campaign. Another piece of the network strategy has just emerged, in an e-mail that just went out on Obama's list regarding a new tool, called "Neighbor to Neighbor." See excerpt:

It's up to each of us to talk to voters across the country and make sure they know what this election is really about.

We have an exciting new tool called Neighbor to Neighbor that makes it easy to talk to potential supporters about Barack and the issues that matter.

Help get the conversation back on track today by making phone calls.

Nothing is more powerful than having undecided voters hear from ordinary people. And right now, that's needed more than ever.

No prior experience is required. Neighbor to Neighbor gives you a list of potential supporters, suggested topics to talk about, and an easy way to report back on who you've contacted.

With less than eight weeks until Election Day, we can't allow voters to lose focus on the big issues and get swept up by the smears and lies coming out of the McCain campaign.

Reach out to fellow voters now and grow this movement for change:

http://my.barackobama.com/n2n

This is based on a pretty strong theoretical understanding of how networks mobilize action; e.g., this is consistent with field experiments on how to increase turnout. I am not familiar with similar field experiments on how these methods affect preference (as compared to turnout). Any cites along these lines would be welcome as comments. (One wonders if campaigns conduct experiments along these lines prior to the election; this would certainly be doable, and a drop in the bucket given overall expenditures.)

This is, in any case, a striking departure from the "mobilize the base" strategies of recent years, where the objective was to get partisans to talk to each other, not partisans to undecideds. And arguably, given the apparent value of intersecting with people with different views, good for our democracy (e.g., see Diana Mutz's recent book).

I would be interested if there have been any surveys with an item like: "Have you spoken to anyone about the election, and what were their candidate preferences?" Does one see a difference in persuasion attempts by Obama as compared to McCain supporters? Are these persuasion attempts targeted at undecideds?

July 21, 2008

Facebook vs MySpace

In David Carr's column "The Media Equation" in today's (7/21/08) New York Times, he writes about the amazing growth of Facebook usage and visits. Facebook does indeed have its social — and business — prerogatives. The network platform is on a tear right now, achieving numerical parity with MySpace in global reach.

As Carr writes:

Last month, according to comScore, Facebook had 123.9 million unique visitors and 50.6 billion page views worldwide while MySpace had 114.6 million unique visitors and 45.4 billion page views. MySpace still dominates in the United States, but if my page is any indication, a lot of people who aren’t texting OMG about the guy sitting in the next booth feel a need to opt in to social media.

According to company executives, Facebook, which has over 80 million subscribers worldwide, doubled the number of subscribers under 35 last year, but it tripled the number of subscribers between 35 and 54. Early adopters of Facebook, which was the province of students until 2006, must wonder who let all the old guys in. Sometime in the next day or so, Facebook will unveil a major new design for the site, which users can opt-in to.

Facebook is indeed on a tear. One wonders what it's now worth ......

July 2, 2008

Live by the netroots die by the netroots?

Interesting article in today's New York Times regarding the resistance Obama is experiencing from his supporters regarding his support of legislation to give immunity to telecommunications companies that cooperated with the Bush administration's wiretapping program. Notably, supporters are using the web-based platform created by the Obama campaign to protest his position. The interesting thing is that in creating a genuine capacity for supporters to collectively mobilize, the campaign has created something with some autonomy from the campaign. Thus, when Obama shifts from where supporters think he stood, there is the real potential for a backlash from those supporters, using the tools for collective action that the campaign created.

Of course, the challenge for Presidential candidates in balancing the base against the center, in moving from the primary to the general campaign is nothing new. However, the speed of the backlash from the grassroots, and the use of the campaign's own mobilization machinery, is novel.

Here are excerpts from the article:


Obama Voters Protest His Switch on Telecom Immunity

By JAMES RISEN

WASHINGTON — Senator Barack Obama’s decision to support legislation granting legal immunity to telecommunications companies that cooperated with the Bush administration’s program of wiretapping without warrants has led to an intense backlash among some of his most ardent supporters.

Thousands of them are now using the same grass-roots organizing tools previously mastered by the Obama campaign to organize a protest against his decision.
In recent days, more than 7,000 Obama supporters have organized on a social networking site on Mr. Obama’s own campaign Web site. They are calling on Mr. Obama to reverse his decision to endorse legislation supported by President Bush to expand the government’s domestic spying powers while also providing legal protection to the telecommunication companies that worked with the National Security Agency’s domestic wiretapping program after the Sept. 11 attacks.

During the Democratic primary campaign, Mr. Obama vowed to fight such legislation to update the Foreign Intelligence Surveillance Act, or FISA. But he has switched positions, and now supports a compromise hammered out between the White House and the Democratic Congressional leadership. The bill is expected to come to a vote on the Senate floor next Tuesday. That decision, one of a number made by Mr. Obama in recent weeks intended to position him toward the political center as the general election campaign heats up, has brought him into serious conflict for the first time with liberal bloggers and commentators and his young supporters.

June 4, 2008

VisiblePath is dead .... Long live Hoover's Connect

As many of you know, VisiblePath, the company I was "involved with" for more than five


VP logo

years, was acquired earlier this year by Hoovers. The business network software is

Hoover's logo

now available as Hoover's Connect.

Another social network software firm disappears ..... It was a fun ride.


May 22, 2008

More on the Social Context of Smoking

In addition to the story by Gina Kolata in The New York Times that David mentioned in his post,

http://wwwimage.cbsnews.com/images/2001/07/17/image301833x.jpg

there is the story by Alicia Chang for the Associated Press, which was picked up by many AP outlets Friends quit smoking? You probably will too.

From that story:

While the study was cleverly done, it does have its limitations.

...... it's hard to tease out whether social influence is mainly responsible for a whole group kicking the habit. Other factors such as public bans on smoking or studies highlighting the harmful effects of smoking may also play a role.

"You can't prove it with this data," he said. "You have to go to people and ask, 'Why did you stop smoking?'"

There's no question that the Framingham data are unique. Let's hope that others will realize that to really study public health issues, you must measure the social contexts that the subjects are embedded in.

Let's hope that others will begin the slow process of gathering data on both public health issues and the proper social networks.

May 21, 2008

The contagiousness of smoking?

Article appearing in tomorrow's New England Journal of Medicine examines the network dimension of smoking:

The Collective Dynamics of Smoking in a Large Social Network

Nicholas A. Christakis, M.D., Ph.D., M.P.H., and James H. Fowler, Ph.D.

Will be interested in seeing how the causal angle was pulled out. In particular, the challenge is that people were being differentially pounded with signals to stop smoking during this period, and that differential exposure is likely correlated with network position.

The re-examination by Van den Bule and Lilien of the classic Coleman study, which found that rather than the social network it was differential exposure to marketing that drove adoption of tetracycline, is a nice illustration of how social networks can be correlated with external influences.

In any case, here is the abstract of the article:

ABSTRACT

Background The prevalence of smoking has decreased substantially in the United States over the past 30 years. We examined the extent of the person-to-person spread of smoking behavior and the extent to which groups of widely connected people quit together.

Methods We studied a densely interconnected social network of 12,067 people assessed repeatedly from 1971 to 2003 as part of the Framingham Heart Study. We used network analytic methods and longitudinal statistical models.

Results Discernible clusters of smokers and nonsmokers were present in the network, and the clusters extended to three degrees of separation. Despite the decrease in smoking in the overall population, the size of the clusters of smokers remained the same across time, suggesting that whole groups of people were quitting in concert. Smokers were also progressively found in the periphery of the social network. Smoking cessation by a spouse decreased a person's chances of smoking by 67% (95% confidence interval [CI], 59 to 73). Smoking cessation by a sibling decreased the chances by 25% (95% CI, 14 to 35). Smoking cessation by a friend decreased the chances by 36% (95% CI, 12 to 55 ). Among persons working in small firms, smoking cessation by a coworker decreased the chances by 34% (95% CI, 5 to 56). Friends with more education influenced one another more than those with less education. These effects were not seen among neighbors in the immediate geographic area.

Conclusions Network phenomena appear to be relevant to smoking cessation. Smoking behavior spreads through close and distant social ties, groups of interconnected people stop smoking in concert, and smokers are increasingly marginalized socially. These findings have implications for clinical and public health interventions to reduce and prevent smoking.


And excerpts from coverage in tomorrow's New York Times:

May 22, 2008
Study Ties Smoking to Social Circles
By GINA KOLATA

For years, smokers have been exhorted to take the initiative and quit — use a nicotine patch, chew nicotine gum, take a prescription medication that can help, call a help line, just say no. But a new study finds that stopping is seldom an individual decision.

Smokers tend to quit in groups, the study finds, which means smoking cessation programs should work best if they focus on groups rather than individuals. It also means that a people may help many more than just themselves by quitting — quitting can have a ripple effect prompting an entire social network to break the habit.

The study, by Dr. Nicholas Christakis of Harvard Medical School and James Fowler of the University of California in San Diego, followed thousands of smokers and nonsmokers for 32 years, from 1971 until 2003, studying them as part of a large network of relatives, co-workers, neighbors, friends, and friends of friends.

...

May 5, 2008

Violence is Contagious!!! Just like Obesity!!!

Did you see this (yesterday's --- May 4 --- NYT Sunday Magazine).

http://graphics8.nytimes.com/images/2008/04/30/magazine/04cover_395.jpg

Is this ridiculous, or what?
Sure, it's a nice story line, but are we to believe that if you hang out with violent people you will become violent?


April 22, 2008

MOMA: Design and the Elastic Mind

An amazing show at the Museum of Modern Art, in NYC --- Design and the Elastic Mind.

Many of the "exhibits" are network representations (including a few that I use in elementary lectures on network science). And the website showing them (since the exhibits are all digital) is amazing.

Check it out: Design and the Elastic Mind .

April 9, 2008

Think Facebooking is a waste of time? Think again...

This hardly comes as a surprise: Corporations are increasingly tapping into the social capital of networks such as Facebook and MySpace, as reported in this NY Times article by Laurie J. Flynn today. From a theoretical standpoint, it makes a lot of sense: The ties in these online social networks reflect several layers of homophily (friendship, common interests, membership in various groups, partially self-selected affiliation, etc.) in addition to what usually applies to even the best organizational communities of practice. Several companies are now integrating business intelligence applications with the social Web and the Internet. Such "interrelated pools of information" bring value to business, says Flynn, mainly by fostering communication among employees, but also by better identifying job candidates and target customers. Let's just hope that Facebook will react to this development and allow the creation of different profiles for the various personae we represent on the Internet.

The article appeared in a special section of the New York Times today called "Tech Innovation". The section is filled to the brim with exciting and innovative ideas - one of these coming from the ever resourceful Bernardo Huberman of HP Labs. Together with his team he developed the prediction markets tool "Brain" (Behaviorally Robust Aggregation of Information in Networks), which can be employed to predict the demand of a new service, such as Internet television. I loved Huberman's quote a propos his brainchild: "We want to reduce the wisdom of crowds to the wisdom of 12 or 13 people." Hopefully the right ones.

March 27, 2008

Finding Political News Online, the Young Pass It On

Interesting story in today's New York Times on how younger voters acquire news. This article highlights (1) the partial disintermediation of traditional media (e.g., because some content comes from sources like Youtube), (2) the increased power of the word-of-mouth second step in the classic two step model of diffusion afforded by e-mail/internet, and (3) how the medium facilitates grassroots mobilization. Excerpts:


According to interviews and recent surveys, younger voters tend to be not just consumers of news and current events but conduits as well — sending out e-mailed links and videos to friends and their social networks. And in turn, they rely on friends and online connections for news to come to them. In essence, they are replacing the professional filter — reading The Washington Post, clicking on CNN.com — with a social one.

....

A December survey by the Pew Research Center for the People and the Press looked broadly at how media were being consumed this campaign. In the most striking finding, half of respondents over the age of 50 and 39 percent of 30- to 49-year-olds reported watching local television news regularly for campaign news, while only 25 percent of people under 30 said they did.

....

The way consumers filter their news is being highlighted now that a generation of Americans is coming of age in the midst of a campaign that has generated intense interest and voter involvement. Exit polls in 22 states estimate that more than three million voters under the age of 30 participated in Democratic primaries this year, up from about one million four years ago.

March 23, 2008

The deliberative presidency

Very interesting piece in NYT last week on the decision by the US to dissolve the Iraqi Army in 2003. An excerpt:

Colin L. Powell, the secretary of state and a former chairman of the Joint Chiefs of Staff, said he was never asked for advice, and was in Paris when the May 22 meeting was held.

Mr. Powell, who views the decree as a major blunder, later asked Condoleezza Rice, who was serving as Mr. Bush’s national security adviser, for an explanation.

“I talked to Rice and said, ‘Condi, what happened?’ ” he recalled. “And her reaction was: ‘I was surprised too, but it is a decision that has been made and the president is standing behind Jerry’s decision. Jerry is the guy on the ground.’ And there was no further debate about it.”

In this election season, perhaps the hardest thing to evaluate about the candidates is one of the most important: how would she or he create a deliberative Presidency? We are in the midst of a paradigm shift in our policies at home and abroad, and the next Presidency will be built brick by brick through the myriad of decisions that need to be made in response to the particular challenges that arise on a daily basis. Those decisions must reflect the President's vision of the world, but through the prism of the information and options that exist at particular instances in time. What does it take for the Presidency to successfully navigate the uncertainties of the 21st century?

Extensive research on individual and organizational decision making, including my own work on networks, highlights one prerequisite to high quality decisions: diversity of perspectives and sources of information. The modern Presidency is made up of a mix of careerists and transient political appointees in the Executive Office of the President, connected to relatively slow changing agencies. The key challenge is one of how to manage the connections among these individuals, in order to balance the simultaneous needs for deliberation and decision. Disagreement offers Presidents a menu of choices, where dissent can keep options open, and forces others to critically examine their own perspectives.

The successful President is one who can manage this complex network, creating a pull of information and viewpoints upwards. Doris Kearns Goodwin, in her study of Lincoln, presents a compelling portrait of a President who filled his cabinet with individuals with strong personalities and often conflicting views. In a rather different fashion, Franklin Roosevelt also structured his network so as to receive multiple perspectives, through privately soliciting information from rivals within the White House.

Recent Presidents have developed more institutionalized mechanisms to funnel information upwards. My colleague, Roger Porter, has documented the collegial, if sometimes adversarial, process of soliciting input from agencies regarding economic policy in the Ford administration. I have studied how the regulatory review processes instituted in the Reagan administration has offered the White House a steady stream of signals about regulatory policy in the Reagan, Bush, and Clinton administrations. Both of these processes took advantage of the natural differences of perspective that are embedded within the bureaucracy. These differences can be used to provide illumination as well as heat.

The President also needs to admit some degree of uncertainty as to the best course of action. No President, of course, has begun a speech "I am 51% certainty that I have made the best choice." But internal discussion must begin with an assumption of uncertainty and a respect for alternative points of view. That deliberative foundation must come from the President.

The general set of issues to balance is how distribute involvement in various decisions. Who has information, perspective, and expertise to contribute to a decision? How does one not clog the decision making network with issues of secondary importance so that the key decisions get the collective attention they deserve?

The history book has not yet closed on the current administration, but insider portrayals suggest a systematic mismanagement of the decision-making network. The decision to dissolve the Iraqi army is an example par excellence. This was one of the fundamental strategic decisions post invasion, and one where different bureaucratic actors had different things to contribute to the decision. Clearly, Bremer was in a privileged position vis a vis some of the information, but that did not mean that he was even the best person to process that information. Nation building generally, and rebuilding Iraq specifically was a domain that no one in the US government could claim such unique expertise as to require such deference. This case, and others like it, suggest a fundamental mis-management of the inter-agency process in foreign policy in the Bush Administration.

There is a potential trade-off of decisiveness and discipline on the one hand, and deliberativeness on the other. Clearly, the costs of dithering sometimes exceed the benefits of discussion. However, we face long run existential threats at home and abroad: with radical changes in the distribution of power in the world, and a restructuring of our economy at home. This next Presidency may well prove to be a pivotal one in history, and, to be successful, will require a network that supplies information and perspectives with which to choose the future.

March 8, 2008

The Machinery of Hope

There was a really interesting article in Rolling Stones on the Obama campaign, The Machinery of Hope (thanks go to Valdis Krebs to pointing it out to me). It highlights some of what I have discussed before-- the creation (top down) by the Obama campaign of an architecture to enable bottom up mobilization. Some excerpts, but I strongly recommend reading the entire article:

Over the past year, the Obama campaign has quietly worked to integrate the online technologies that fueled the rise of Howard Dean —as well as social-networking and video tools that didn't even exist in 2004 — with the kind of neighbor-to-neighbor movement-building that Obama learned as a young organizer on the streets of Chicago.

The meeting in San Marcos wasn't advertised in any traditional sense. Instead, the campaign posted the event on my.barackobama.com — its social-networking site affectionately known as "MyBo" — and e-mailed local residents who had donated to the campaign or surrendered their addresses as the price of admission to an Obama rally. And the volunteers who showed up won't be micromanaged…. They'll be able to call their own shots, from organizing local rallies to recruiting and training a crew of fellow Obama supporters to man their precincts on election day. To identify and mobilize Obama backers, they'll log on to the password-protected texasprecinctcaptains.com, download the phone numbers of targeted voters, make calls from their homes and upload the results to Austin headquarters. They'll also organize early-voting open houses — which will be publicized on MyBo — to boost turnout among core supporters. "Instead of hoping that your neighbors vote," Ukman tells them in an unintentional twist on the campaign's central theme, "you're going to take them to the polls."

Hildebrand actually flipped the equation, using the physical crowds Obama could draw to his rallies to bolster the campaign's e-mail list. In February and March of 2007, just after Obama announced his candidacy, the campaign set up huge rallies in cities from Los Angeles to Austin to Cleveland. In return for a ticket, supporters were asked only to provide their e-mail, zip code and telephone number

In Iowa, as many people under thirty caucused as did senior citizens. In every contest, the youth vote has at least doubled and often tripled previous records. Riemer is quick to point out that these successes aren't just the result of the campaign organizing young people but of young people organizing themselves.


Figueroa's goal is not to put supporters to work but to enable them to put themselves to work, without having to depend on the campaign for constant guidance. "We decided that we didn't want to train volunteers," he says. "We want to train organizers — folks who can fend for themselves."

To turn well-meaning students and nurses and social workers into self-sufficient organizers, the campaign has put nearly 7,000 supporters through an intensive, four-day seminar known as "Camp Obama."

A strategy that leans so heavily on the grass roots is not without risk. In February, right-wing blogs had a field day when a Fox News affiliate ran footage of a volunteer office in Houston decorated with a Che Guevara flag.

February 29, 2008

How To Use LinkedIn

There is a great piece in yesterday's New York Times (28 February) by Michelle Slatalla on using LinkedIn and similar network sites to actually make business connections. In Building a Web of Influence, she writes:

My problem could be networking. Or more specifically, a lack of it. I work in a basement where my only business contacts are my dogs, which appear unimpressed by my résumé. And on rare occasions when I venture aboveground to attend an event with the sort of people who should be only too willing to offer stock options in return for my grandmother’s chocolate cake recipe, I get tongue-tied. I blush.

On the Internet, however, no one can tell you’re self-conscious. Business networking sites — from the five-year-old site LinkedIn.com to an upstart called NotchUp.com — attract members by stating that reluctant self-promoters like me can make contacts, and even get job interviews, fairly passively.

she goes on ....

Slowly, I began to see concrete signs of the value of passive networking. One day, a first-degree contact asked me to introduce her through LinkedIn to one of my other first-degree contacts, a writer at The New York Times who my friend thought might be interested in a book she wrote. She phoned soon after to say thank you; the writer had responded right away.

A few days later, the same newspaper writer coincidentally forwarded an introduction to me from one of her first-degree contacts, a magazine editor looking for a humor writer.

Meanwhile, another first-degree contact told me her daughter had landed an interview after learning at LinkedIn that the interviewer and she went to the same college.

and concludes:

In my case, I’m too busy now to change jobs. But someday, after my dogs learn to turn doorknobs with their paws and my husband’s cough finally breaks up, I will be ready.

I only hope that by then, the Exxon recruiter I see listed among my third-degree LinkedIn contacts will be searching for someone with expertise in “weekly allowance arbitration.”

It's a great piece. Take a look at it.

February 24, 2008

Obama, techology, and social networks

There is a very nice piece in today's Boston Globe on the role that technology has played in mobilizing support for Obama. There are snippets below, but I recommend reading the whole article.

There are a few key ingredients, it seems to me, to the strategy. First is the facilitation of the meeting of people with shared interests and goals. Second is creating a forum for people to express their views. Third is the provision of scripts and plans for the mobilization and coordination of action.

I should note that this is a combination of a bottom-up and top-down processes. This is not really like Meetup.com facilitating the rise of Howard Dean. The engineering of the site my.barackobama.com was designed to achieve certain, very specific, ends. It is notable that the Obama campaign developed its own proprietary platform rather than rely on existing platforms, such as Meetup and Facebook (although, Obama very much has a presence on those websites as well). This was, in part, for control reasons, I assume. The provision of those plans is also clearly a top down process. However, the actual use of these tools is totally bottom up, where the decisions of people to join the website are interdependent, enthusiasm breeding enthusiasm.

The success of the Obama campaign highlights one of the shortcomings of one of the dominant perspectives on political mobilization. Following from Mancur Olson's classic book, The Logic of Collective Action, it is a puzzle why people would contribute so much time and effort when, individually, their work is extremely unlikely to be decisive. This view of collective action is based on an assumption that the reason why people act is based on the impact their individual efforts will make to the collective endeavor. In fact, humans are social animals, and it is that combination of connection (with others) and cause that is so very powerful in politics and other domains (most notably, religion).

In the comments I would be interested in hearing the observations of others about the role of networks and technology in this political season.

In any case, the excerpts:

Technology aids Obama's outreach drive
Volunteers answer call on social networking site

More than any previous presidential campaign, Obama's effort is transforming politics with its use of technology. The astounding fund-raising figures are well documented - the campaign keeps a running tally on its website as it closes in on 1 million donors. But Obama's team has taken the use of the Internet to another level by allowing masses of volunteers to self-organize over the past year and communicate through their own social networking site, my.barackobama.com.

Created with help from Chris Hughes, one of three Harvard roommates who invented Facebook four years ago, MyBO, as campaign staffers call it, has about 500,000 members nationwide, a network of groups and individuals that the campaign ultimately harnesses for the old-fashioned nuts-and-bolts of electioneering – identifying supporters and getting them to vote in primaries and caucuses.

Clinton's campaign, relying more on traditional resources such as labor unions and elected officials, is also cobbling together an Ohio organization. Using its own e-mail list, the Clinton team recently received about 1,000 online replies statewide from people willing to help. The campaign was pleased with the response. The Obama camp, in contrast, drew 500 people from the Columbus area alone to the union hall of Plumbers & Pipefitters Local 189 on a Wednesday night in the middle of winter.

...

Valli Frausto, a 50-year-old mother of two from Columbus, was one of the first to open a MyBO account when Obama announced his candidacy a year ago. Several months earlier, she had seen Obama on Oprah Winfrey's show.

"I've never been involved in a political campaign before, but it was like a call to action for me," she said. "I said if he runs, I want to help, and with the way he put his campaign together, with all these tools available to us, it allowed me to get involved."

For the past year, Frausto estimates she has spent 20 hours a week as one of the administrators of Central Ohioans for Obama and a few others of the more than 300 groups established in Ohio through MyBO.

...

The groups have put out information tables at perhaps 20 festivals and fairs around Columbus, and held a 5-kilometer road race and other fund-raisers. They also hold debate parties, phone bank events, and happy hour gatherings to socialize, brainstorm, and introduce new members.

"It was all done through my.barackobama.com," Frausto said. "We would not exist if not for that tool. It's phenomenal to me."

Now, however, with the Ohio primary approaching, the campaign is much more actively coordinating the activists, most notably through phone banking. From a MyBO page, a member can click onto a list of 20 phone numbers with a series of prompts and scripts that the caller runs through, entering the responses of voters online. The information goes into the campaign database for its primary day get-out-the-vote operation.

February 19, 2008

The bigger your network, the better your outcome ....

From today's (2/19/08) NYT Science section ..... Large social networks may help surgical patients .... it appears that having a large network of friends and family may help you do better in surgical procedures (surprised?).

google%20trends.png

From a study published in the February 2008 issue of The Journal of the American College of Surgeons,
Social Connectedness and Patient Recovery after Major Operations, one of the authors, Daniel B. Hinshaw, a professor of surgery at the University of Michigan , states:

"The average physician, when he takes a social history, asks about smoking and drugs and not the real social situation of the patient. And yet it looks like this is a real marker for problems".

February 5, 2008

Jennifer Chayes

Jennifer Chayes, of Microsoft Research (formerly in Math at UCLA) was just named director of a new Microsoft Research Lab in Cambridge.

She is well-known to the Network Science crowd, doing good work in statistical physics. She also knows social networks. Good choice by Microsoft. Nice piece in yesterday's New York Times: Microsoft Adds Research Lab in East as Others Cut Back.

From the article:

The work she did in developing simple models of certain liquids and solids turned out to be useful in the study of random, self-engineered networks like the Internet. And some of Dr. Chayes’s insights into theoretical computer science have recently led to the development of some exceedingly fast networking algorithms.

January 19, 2008

Information, networks, and markets

It is notable, if not surprising, that the best place, in all of the Internet, to go for a sense of the direction that an election is going is intrade.com. Intrade is a futures market, where you can buy and sell contracts that pay off depending on the outcome of some event in the future. So, what has happened in the primaries is that that the futures market shows substantial shifts through the day of the event. Today is typical, where hours before poll closings, the market for the Republican race South Carolina swung hard toward McCain (who apparently is going to win).

What is interesting is that, as far as I can see, these markets swing before any other information is available on the Internet. That is, the market is being driven by those who presumably have a close connection to inside information—i.e. exit polls. This is not to say that employees of CNN are taking time off during the day to put money into intrade contracts, but that there is an informal leakage of information from those in the know, where that leakage then drives the market. That information can be misleading, of course (cf. Kerry, 2004); but the market allows for that possibility (e.g., even now Huckabee is trading at 2%).

This is an interesting natural experiment of sorts, because there those exit polls are likely by far the most reliable sources of information available regarding election outcomes (especially for primaries) before actual results come in. There are other bits of information that may be independently valuable, like any data on differential turnout across regions, etc, but generally the exit poll data are going to dominate everything else. It would therefore be interesting to calibrate the actual data that exit poll entities have gathered at what points in time (minute by minute) to see how quickly intrade responds. This should provide some sense of the speed of informational leakage from the exit polls. (And if you know of anyone has already done this, please post a comment.)

Btw, this graph shows the shift in the value of the McCain winning SC contract:

mccain%20sc.gif

Note that the time marked is GMT, so the major shift occurred at around 6pm EST (exit polls were not released publicly until after 8pm).

January 14, 2008

Huck's network

Interesting piece in yesterday’s New York Times that highlights the interplay of old and new ways of organizing, vis a vis the grassroots efforts for Governor Huckabee. A few notable points to highlight: (1) how the Internet facilitates bottom-up collective action in a way that would have been much more difficult a decade ago; (2) the emergent collective action builds on pre-existing evangelical networks; and (3) how the technology, in particular, potentially disintermediates the old guard leadership. That is, the old modality of collective action would have been more top down, reflecting the privileged informational position of the old guard, who had built up mailing lists and the like that enabled them to mobilize their constituencies. With the Internet, their constituencies can now find each other.

Excerpts from:
Huckabee Splits Young Evangelicals and Old Guard


WASHINGTON — Much of the national leadership of the Christian conservative movement has turned a cold shoulder to the Republican presidential campaign of Mike Huckabee, wary of his populist approach to economic issues and his criticism of the Bush administration’s foreign policy. But that has only fired up Brett and Alex Harris.

The Harris brothers, 19-year-old evangelical authors and speakers who grew up steeped in the conservative Christian movement, are the creators of Huck’s Army, an online network that has connected 12,000 Huckabee campaign volunteers…

In Michigan, the Huckabee campaign had spent no money, hired no staff and had no office until last Wednesday, six days before the primary. But Gary Glenn, a conservative Christian advocate based in Midland, Mich., has been leading an informal effort to turn out evangelical voters. Some pollsters expect them to make up as much as 40 percent of the state’s primary voters this year.

Last week, Mr. Glenn lined up 50 local pastors to attend a closed-door breakfast with Mr. Huckabee in Grand Rapids. And he has compiled an e-mail list of more than 600 volunteers — many in Internet groups that Huck’s Army is connecting — who have been using church directories to make phone calls, courting local pastors and leafleting church parking lots .
“Recruit volunteers to stand this Sunday on public sidewalks across the street from the parking lots of the biggest evangelical churches you can find,” Mr. Glenn urged in a recent e-mail message.
….
[In South Carolina]… more than 500 people, many of them young evangelicals, have signed up for online Huckabee meet-up groups, said Christian Hine, 30, the state coordinator of the Huck’s Army effort. Unaided by the campaign, volunteers have borrowed church directories and bought their own phone lists to try to identify likely Huckabee voters, Mr. Hine said, and even paid to print their own Huckabee signs when the campaign ran out.

January 3, 2008

More on Iowa networks...

Let me make a few observations about the results tonight, based in part on the CNN entrance polls. First, the evangelical network beat the machine on the Republican side, with evangelical turnout making up about 60% of Republican voters, with Huckabee winning evangelicals decisively, and Romney winning non-evangelicals. Second, on the Democratic side (especially), turnout was the story, with a close to doubling of turnout from 2004. Particularly remarkable was the strong turnout of young voters—22% of those who turned out on the Democratic side were 17-29 (compare to 11% in Republican caucuses)—the same percentage as those 65 and over. Clinton decisively won those 45 and over (who still made up 60% of voters), but Obama just overwhelmed Clinton among those 45 and under.

In any case, the point here I make (casually) is that the individual decisions how to vote and whether to vote are not independent observations—that there is a pre-existing social organization (evangelicals in church, students in college)—which serves as the vessel for political action. I don’t claim this as a novel observation about politics, just one that is highlighted in these numbers.

Iowa networks...

Happy new year all! And for those political junkies out there, happy Iowa caucuses.

Has there been any work on the Iowa caucuses and social networks? There are a multitude of things one could study, notably around mobilization and caucusing choice. There has been more general work on networks and opinion (e.g., Huckfeldt, Sprague, etc), and networks and political mobilization (e.g., Nickerson).

Particularly interesting in 2008 is the contrast in mobilization strategy on the Republican side between Huckabee and Romney, where Romney has the traditional (well funded) mobilization machine, with phone calls, mass mailing, rides, etc; whereas Huckabee (out of necessity) is relying on informal evangelical networks to mobilize his supporters. Conventional wisdom is that the machine versus network match up will yield a several point edge to Romney.

---------------------
An interesting potential study: to examine the pre-caucus preferences (e.g., as measured by CNN entry poll) and post-caucus vote distribution.

Another potentially interesting study: examining the micro-level interactions within some of the caucuses.

In any case, if you know of relevant papers/research (or news about today's caucuses), please post as comment.

December 20, 2007

Silicon valley networks

Another article in today’s New York Times evoked social network ideas, on the role of networks in Silicon Valley. (It is a sign of the times that I have blogged about three New York Times articles this week that were somehow about networks). This is a topic, of course that has received ample scholarly attention.

Some excerpts:

Yet a look at the microclusters within Silicon Valley demonstrates the business relationships, the social connections and the seamless communication that animate the region’s economy. It also suggests the human nuance behind the Valley’s success and shows why that success is not easy to copy, export or outsource.

Then there are the clusters that are based more on personal connections or affiliations than on geography. Stanford, just outside Palo Alto, is perhaps the strongest cluster-generator in the Valley.

Stanford students and staff have been behind countless companies in the Valley, from established ones like Hewlett-Packard, Cisco and Sun Microsystems to more recent success stories like Google and VMware.

New companies with deep ethnic links — mainly Indian and Chinese — are sprouting up in the Valley. Often, ethnic background is but one layer of social relationship. SnapTell, a start-up that seeks to marry image-recognition technology, cellphones and marketing, was founded last year by G. D. Ramkumar and Gautam Bhargava, Indian computer scientists and Valley veterans. The company has 10 employees, six of whom have Ph.D.’s and three of whom are from Stanford.
The shared backgrounds, interests and schools make for frictionless communication that fosters rapid innovation.

Even weekend sports, it seems, become the basis for informal business clusters in the Valley. Start-up ideas or job opportunities often surface on the sidelines of a weekend soccer game or, increasingly, cricket match. Giriraj Vengurlekar, an engineer who lives in Sunnyvale, plays in one of the Valley’s cricket leagues, which now has 40 teams. His team, the Centurions, includes employees of Sun Microsystems, eBay, Cisco, Yahoo and other technology companies.

Last year, Mr. Vengurlekar joined Serus, a start-up that makes software for managing offshore manufacturing operations. The cricket pitch, he finds, is a good place to scout recruits or learn of job openings. “People don’t play cricket to get jobs, but it definitely happens,” he said. “Cricket definitely spills over into business.”

I would note that you never play cricket alone.

It is not surprising that identity and background play a key role in silicon valley networks. The statement The shared backgrounds, interests and schools make for frictionless communication that fosters rapid innovation., in particular caught my eye. The potential downside, one would think, is that these homogeneous clusters do not foster the innovation that is the result of different, complementary, backgrounds; of the recombination of existing ways of thinking (cf Reagans and Zuckerman 2001 on the role of diversity on teams).

Ray Reagans, Ezra W. Zuckerman, Networks, Diversity, and Productivity: The Social Capital of Corporate R&D Teams, ORGANIZATION SCIENCE, Vol. 12, No. 4, July-August 2001, pp. 502-517

baseball networks

Interesting article in New York Times last Sunday on spread of use of performance-enhancing substances. The punchline:

Mitchell’s report contended that the use of performance-enhancing substances was spread not by stars, but by journeymen like Segui.

The report described how as Segui traveled, he put teammates in touch with Radomski, who as a former Mets clubhouse employee was viewed as an insider. Frequently, the referrals to Radomski as depicted in the report were to marginal or injured players trying to keep their jobs.

There was a neat graphic illustrating the role that Segui played in the spreading process. This is reminiscent of studies out there examining the early spread of HIV. In short, journeymen play a key bridging role in the system in disseminating information. It would be interesting to plot average degree of a player (as measured by number of teammates over the years) and probability of being named in Mitchell report, as well as the probability of being a conduit for the spread of usage more generally.

Have there been studies that have looked at player "networks" (based on who has been teammates with whom)? I guess this is formally similar to the work that Uzzi and collaborators have done on Broadway productions; one could easily produce the same kind of graphs for any sport. Not sure there are interesting questions that one could answer with data like these, but perhaps.... I wonder what role such bridges may have played, for example, in the emergence and effectiveness of unions in sports?

December 19, 2007

Datamining facebook

Apropos computational social science, there have been a couple of articles on using facebook for social science research, one on Sunday in the Washington Post, and one on Monday (front page) in the New York Times.

An excerpt from the New York Times article:

To study how personal tastes, habits and values affect the formation of social relationships (and how social relationships affect tastes, habits and values), a team of researchers from Harvard and the University of California, Los Angeles, are monitoring the Facebook profiles of an entire class of students at one college, which they declined to name because it could compromise the integrity of their research.

“One of the holy grails of social science is the degree to which taste determines friendship, or to which friendship determines taste,” said Jason Kaufman, an associate professor of sociology at Harvard and a member of the research team. “Do birds of a feather flock together, or do you become more like your friends?”

In other words, Facebook — where users rate one another as “hot or not,” play games like “Pirates vs. Ninjas” and throw virtual sheep at one another — is helping scholars explore fundamental social science questions.

We’re on the cusp of a new way of doing social science,” said Nicholas Christakis, a Harvard sociology professor who is also part of the research. “Our predecessors could only dream of the kind of data we now have. [emphasis mine]

Facebook is an example par excellence of the emergent media that facilitate communication, but also leave a wonderful record for researchers to mine. The interesting question is how to design research so as to pull apart exactly the causal conundrum Kaufman mentions (the coevolution problem I have written about). How does one pull apart shared histories (which may be correlated with the network) from diffusion, for example? Observing, for example, things diffuse through facebook does not tell us, necessarily, that it is the network driving that diffusion, because there may be shared exposure to things like marketing. In any case, the facebook offers new vistas in terms of pulling these things apart.

November 7, 2007

The other side of social networking

by David Allen

The Financial Times recently reported on the rise of bullying - and hate mongering - as web counterparts to 'positive' social networking. As a sample, the story's sidebar describes three 'counter' sites:

Enemybook_Goes under the strap line "Keep your friends close, and your enemies closer". Set up as a riposte to the perceived bogus nature of many online friendships, Enemybook runs off the back of Facebook. It allows you to add people as Facebook enemies below your friends, specify why they are enemies and notify them that they are enemies. You can also see who lists you as an enemy, and even become friends with the enemies of your enemies.

Snubster_Similarly to Enemybook, Snubster derides the notion of social networking sites, and can run off Facebook. Users can build lists of personal enemies from their Facebook contacts, who will then be sent a snub and will be alerted that they are either "On notice"
or "Dead to me".

Hatebook_Modelled on the Facebook concept, and with an almost identical layout, Hatebook offers a less friendly approach to the world of social networking. You can befriend "Other haters", and your homepage alerts you when "Other fricking idiots" contact you. The site also provides you with an "Evil Map", marking the locations of other users. The antithesis to Facebook's emphasis on making friends, this is an open forum for abuse and aggression.

October 16, 2007

Nature editorial-- A matter of trust: Social scientists studying electronic interactions must take the lead on preserving data security

Nature had news article and an accompanying editorial in its most recent issue on the issues around privacy in the developing field of computational social science. These pieces do a nice job of highlighting the need to develop a powerful institutional infrastructure to facilitate the growth of computational social science. If I were to point to one place where this burgeoning field, which has enormous potential to do good, could trip up, it would be in dealing with privacy. The key challenges are balancing the need for privacy of the data of individuals, with the benefits of improved knowledge about human behavior. This balance is manageable, but requires a lot of work by social and computer scientists, as well as the existing self-regulatory systems of the research world (e.g., IRB's). Something that I will write about in the future, but below are excerpts from the article and editorial.

Excerpts from news article:

The hottest growth area in the field [of social science] is computational social science. This is often based on privileged access to electronic data sets such as e-mail records, mobile-phone call logs and web-search histories of millions of individuals. Such studies are ushering in a revolution in the social sciences, specialists say. But there is a trade-off between the scientific interest in working with such data and concerns about privacy. “It’s a huge issue,” says David Lazer, a researcher at the John F. Kennedy School of Government at Harvard University.

...

This work [referring to work by Jon Kleinberg that illustrates the potential to "de-anonymize" network data] reinforces the need for a systematic, institutional approach to improving the privacy rights of those whose data are used, says [Marshall] Van Alstyne [of Boston University]. That echoes the conclusions of a May study by the US National Academies, which said that safeguarding privacy cannot safely be left to individual researchers. It stated that: “Institutional solutions involve establishing tiers of risk and access, and developing data-sharing protocols that match the level of access to the risks and benefits of the planned research.” But [Myron] Gutmann [of the University of Michigan and co-author of the study] and other social scientists also stress that the risks should be kept in perspective. Scientists must meet strict rules on any research on human subjects. In contrast, private firms are largely free from such constraints, and already have wide latitude to snoop on, and data mine, their employees’ work habits.


Excerpts from editorial:

For a certain sort of social scientist, the traffic patterns of millions of e-mails look like manna from heaven. Such data sets allow them to map formal and informal networks and pecking orders, to see how interactions affect an organization's function, and to watch these elements evolve over time. They are emblematic of the vast amounts of structured information opening up new ways to study communities and societies. Such research could provide much-needed insight into some of the most pressing issues of our day, from the functioning of religious fundamentalism to the way behaviour influences epidemics....

But for such research to flourish, it must engender that which it seeks to describe. And so it is encouraging that computational social scientists are trying to anticipate threats to trust that are implicit in their work. Any data on human subjects inevitably raise privacy issues (see page 644), and the real risks of abuse of such data are difficult to quantify. But although the risks posed by researchers seem far lower than those posed by governments, private companies and criminals, their soul-searching is justified. Abuse or sloppiness could do untold damage to the emerging field.

September 19, 2007

Finding those mutual friends

There has been a spate of stories (also this story from globe) recently about the use of cell phones to track the locations of your friends. There has also been some talk of linking data from the social networking sites to cell phones, so, for example, you could walk into a room and instantly find someone who you had never met, but who was a friend of a friend (through facebook or some other database that was stored in the phone).

These are clever ideas, but let me throw out another idea: a program for phones that facilitates those “you grew up in Long Island? Did you know John Smith, by any chance?” moments. (A small digression—this exact question (except for the name) was posed to me once. And, surprisingly, I actually did know the person in that case.)

The basic idea is quite simple—if you are talking to someone, and you both have the program, have the phones link via Bluetooth and:

1) find overlapping phone numbers and report them back. A more extensive version of this would also match any incoming or outcoming calls the phones have ever made.

2) collect and match structured data about the owners of the phones—where and when you went to school, where you’ve lived different years, etc., and report back matches. This is all of that information that people initially exchange when they meet. While no substitute, this could be faster and more thorough way to those “Do you know” questions. That is, you would both instantly find out if you both happened to live in Ann Arbor for the same two years in the 1990s.

This would not be a hard program to write, but there is the classic chicken-egg problem of how to get enough people to sign on to make it work. Not my problem—but if you write such a program, let me know.

September 17, 2007

Offline networking course for the Facebook generation

While the baby boomers are slowly taking to online networking (see my earlier post here), some youngsters should probably do a little less of it.
Apparently too much use of Facebook does not have a positive effect on "real-life" (or traditional) networking. An article by Michael Schulman in this week's edition of the New Yorker talks about an NYU freshman seminar entitled "Facebook in the Flesh". The aim of the seminar, part of a series of seminars during freshman orientation, was to re-introduce the Facebook generation to face-to-face meetings. Participants were given a few questions ("What drew you to NYU?") they had to ask each other in pairs. This excerpt from the article nicely summarizes the outcome:

"[The facilitator] blew a whistle. 'Thoughts? Feelings? Reactions?' he said. 'Was it hard?' 'Harder than Facebook,' one girl said."

Some food for thought.

September 13, 2007

"Older people are sticky" - social sites for baby boomers

An article in today's New York Times discusses the emergence and popularity of social networking sites aimed at the 55+ population. Very interesting, I thought, in particular the hypothesis that while these sites might take longer than myspace or facebook to reach high levels of usage, baby boomers are likely to "hang around". This resonates well with investors...read the full article below.

The Graying of the Web
By MATT RICHTEL

SAN FRANCISCO, Sept. 11 — Older people are sticky.

That is the latest view from Silicon Valley. Technology investors and entrepreneurs, long obsessed with connecting to teenagers and 20-somethings, are starting a host of new social networking sites aimed at baby boomers and graying computer users.

The sites have names like Eons, Rezoom, Multiply, Maya’s Mom, Boomj, and Boomertown. They look like Facebook — with wrinkles.

And they are seeking to capitalize on what investors say may be a profitable characteristic of older Internet users: they are less likely than youngsters to flit from one trendy site to the next.

“Teens are tire kickers — they hang around, cost you money and then leave,” said Paul Kedrosky, a venture capitalist and author of the blog “Infectious Greed.” Where Friendster was once the hot spot, Facebook and MySpace now draw the crowds of young people online.

“The older demographic has a bunch of interesting characteristics,” Mr. Kedrosky added, “not the least of which is that they hang around.”

This prospective and relative stickiness is helping drive a wave of new investment into boomer and older-oriented social networking sites that offer like-minded (and like-aged) individuals discussion and dating forums, photo-sharing, news and commentary, and chatter about diet, fitness and health care.

Continue reading ""Older people are sticky" - social sites for baby boomers" »

September 12, 2007

Campaign contributions and the network of hedgers

I’ve blogged in the past about the fund raising networks underlying the campaign. Well, one of my students last term, Katie Selenski, did a nice paper examining the pattern of campaign contributions to Giuliani, Obama, and Clinton. She examined contribution patterns as two mode (contributor x candidate) data. In particular, who maxed out in contributing to at least two of those three candidates? 344 individuals fell into that category in the first quarter of this year (clearly people with more money to spare than I do). To give a flavor, below is a list of California Obama-Clinton hedgers (88 in all).

clinton%20obama%20ca%20hedgers.GIF

co%20key.GIF

While there are some big name entertainers on the list (see Streisand, Spielberg, Manilow, etc), there is a remarkable number of top executives (big circles)—although we don’t know how this compares to nonhedgers. One suspects that for top executives there is a particular instrumental value to these contributions, which is why we would see so many.

Below is the picture of hedgers nationally.

hedgers.GIF

Again, there are a large number of top executives. And, unsurprisingly, there are more Clinton-Obama hedgers than Giuliani-Obama or Giuliani-Clinton. But there are still a remarkable number of folks in the latter two categories. Some interesting patterns: there are more attorneys who do a Clinton-Giulianai hedge than Obama-Giuliani, and more people from finance who go the Obama-Giuliani path. Presumably this reflects the underlying social structure of campaign contributions—e.g., one imagines that many individuals are recruited to contribute by associates, and may have associates who belong to different candidate camps.

In any case, these are nice illustrations of the potential power of these data. I have seen a fair amount of news reporting based on these data—would be interested in postings on academic or professional (i.e., campaign fundraising) uses of these data.

August 4, 2007

Are you in my network?

Interesting article in the NYT this morning: it seems as if the business strategies of cellphone networks have an impact on social networks. People who are in the same network talk more to each other, than people who don't have the same cellphone network.

The article explains how in informal friendship networks the frequency and duration of cellphone calls is lowered as soon as one of the participants switches to another network and that business acquaintances become "friends" through longer and more frequent phone calls when they are in the same cellphone network.

They refer to research on cellphone use being conducted at the Universities of Notre Dame and Michigan. I am wondering if people here at the Media Lab have found out a similar connection: a question for our bloggers Ben and David.

I had not thought about my own personal cellphone usage in this way, mainly because I am not checking how many minutes I have left. From a research standpoint, is your cellphone network/provider really powerful enough to influence the duration and frequency of interactions with people you do not consider your friends and only talk to on purely professional topics?

June 29, 2007

Social-network sites give businesses ideas for new collaboration

Interesting article in today's Boston Globe on the development of Web 2.0 style tools to be used within organizations. The first two paragraphs:

If vendors have their way, a second wave of collaborative technologies -- including wikis, blogs, videos, and mashups, platforms and features borrowed from social-networking sites like MySpace -- soon will wash onto computers in the workplace.

Such interactive technologies are part of a larger trend known as Web 2.0 that has taken root in the consumer space. They have the potential to transform commerce, simplifying communications between employees, suppliers, and customers, speakers told business and information technology professionals at the Enterprise 2.0 conference at Boston's Westin Waterfront hotel last week.

---
It's an interesting question how transformational these kinds of tools could be within an organization. Organizations, obviously, have other mechanisms for information sharing and community building, and thus there may not be the same type of vacuum that one sees in other settings. Further, many of the barriers to information sharing many be institutional and not technical (actually, I am doing some research on this with fellow bloggers Ines Mergel and Maria Binz Scharf). But I do suspect that there will be some cases where these tools could align pretty well with an organization; the trick will be to identify the appropriate settings.

June 9, 2007

The small world of investing

Interesting NBER paper discussed in today’s NYT, by Laren Cohen, Andrea Frazzini, and Christopher Malloy. The gist of it: that investments by mutual funds are more likely in companies in which the manager of a fund has classmates on the board of directors, and, further, the returns on those investments are far higher than investments in which the manager does not have “old school ties.” Further, the timing of these investments is tied to particular news events from the corporation, suggestion some “information transfer” is occurring.

This is both a neat bit of research, which makes a persuasive case for the presence of information transfer. The magnitude of the effect is particularly striking when you consider that their proxies for a social network, while reasonable, are certainly noisy. That is, while attending the same school, especially in the same cohort, is certainly strongly predictive of a relationship, people do not have relationships with all of their classmates, and have many/most of their relationships with non-classmates. So this effect (almost double the return for investments associated with ties as compared to those not) is likely a fairly conservative estimate of the impact of social ties.

I would also note that this paper is also illustrative of entry of social network ideas into economics (generally by a new cohort of scholars), as well as (to revisit my earlier discussions about computational social science) the potential to mine the growing mountains of relational data that now exist in various archives.

June 6, 2007

Friends of my friends are my… enemies?

It will be interesting to observe the impact of Fred Thompson’s incipient entry into the race for the Republican nomination. One part of the story will be the competition for voters. Another will be the competition for dollars. Per earlier entries on the election, a key to mobilizing resources for a run at the Presidency is the network of big money donors. Interestingly, the competition should be particularly brutal among candidates who are in a similar position (“structurally equivalent”) in the fundraising network. If candidates A and B both are friends with C, then A and B are in competition for C’s dollars. The empirical question, in Thompson’s case, is who is he most structurally equivalent to. My naïve guess is that among the big three, McCain would be, with both drawing on a conservative Washington financial base. Giuliani and Romney likely both have fairly structurally distinct financial foundations. In any case, we should be able to tell more by looking at the second quarter financial reports. It also would be interesting to examine who has contributed to more than one candidate, and which candidates tend to draw from the same part of the network.

May 22, 2007

Connections: The Nature of Networks New Science Exhibition, NY Hall of Science

The New York Hall of Science is the host of the 2007 NetSci conference and has also an ongoing exhibition on social networks. It shows kids hands on how social networks are established or fall apart.

It’s a very cool exhibition: the creator Stephen Uzzo has just walked us through it without explaining the different parts. Everyone had to experience for themselves how connections on the Internet are created for example, how a hub connects to the closest spokes around him (we were all standing on a light show type of board, where each of us was a node and when you move around the connections are changing, depending on the shortest path). There were also real spiders creating a spider web.

I will add more on this during the conference and hopefully some pictures...

May 17, 2007

Watts to Yahoo

Notable news in the world of social network research, that Duncan Watts is being hired away from Columbia University by Yahoo.

Duncan Watts, by the by, is a two time speaker on the Cambridge Colloquium on Complexity and Social Networks, and, of course, the Watts of Watts and Strogatz (1998), which marked the beginning of the current era of research on networks by physicists, mathematicians, and the like.

I am not sure what were the key factors in this move, but it is surely notable that one of the "rock stars" of the field is leaving academia. Three obvious potential factors are: (1) the substantial availability of resources in the private sector in this area at this time; (2) the access to proprietary data; and (3) an institutional milieu (at least at Yahoo and Google) more encouraging of cross-boundary innovation than much of academia.

Is this a blip or a wake up call to the academy? I will revisit in a year or so....


D. J. Watts and S. H. Strogatz. Collective dynamics of 'small-world' networks, Nature, 393:440-442 (1998).

May 11, 2007

Social network website reaches a hire level: LinkedIn uses its own who-knows-whom tools to recruit a CEO.

This story (excerpts below) has a nice, recursive, quality.

Social network website reaches a hire level
LinkedIn uses its own who-knows-whom tools to recruit a CEO.
By Alex Pham, Times Staff Writer
May 7, 2007

Dan Nye landed a job as chief executive of a hot Silicon Valley company without even dusting off his resume.
....
Nye was an executive vice president at Advent Software Inc. when Reid Hoffman, chairman of social-networking company LinkedIn Corp., came calling. Hoffman hadn't found him through a headhunter or a classifieds site but through LinkedIn's vast who-knows-whom online network.
,,,
Hoffman had been looking for someone to run the Palo Alto company he founded, which is like MySpace.com for professionals — people can fill out profile pages, then connect for sales leads, expertise or job prospects. A venture capitalist he knew recommended Nye.
...
Not until after he was hired did Nye discover that Hoffman had made dozens of reference checks — without asking Nye for a single name. He found them through LinkedIn.

All Hoffman had needed were the names of the companies where Nye had worked and the years he was there. Hoffman ran that information through LinkedIn's member profiles, finding dozens of people who had overlapped with the prospective hire (members can also search by college attended and by job title). He fired off e-mails and phone calls to numerous people and talked extensively with 27 of Nye's former colleagues.
...
As he weighed the job, Nye turned the tables on his recruiters. He studied up on LinkedIn through its own online tools.

"I was trying to decide whether I wanted to be part of this company," he said.
...
"We now live in a world where it's just not hard to find out about people," [Nye] said.

May 6, 2007

Clinton v Obama: battle of the fund raising networks

Interesting article in today's Boston Globe about the competition for dollars between Obama and Clinton in Massachusetts. Of particular relevance to this blog is the following quote:

"You don't give to causes, you give to friends," said mystery writer Robert Parker of Cambridge, who contributed $4,600 to Obama. "Larry Tribe asked me to contribute to Obama, and so I did." But Parker added that while he would like to see Obama win the nomination, "I'm not an enthusiastic political supporter of anyone."

This highlights the fact that contributions are, in part, mobilized through social networks of like-minded people. It would be interesting to examine the contribution data as 2-mode network data (candidates x contributors) to see (1) how contributors are connected; and (2) how candidates over time are linked to each other through their contributors.

March 30, 2007

60 Minutes story on familial searching

FYI, this Sunday at 7pm CBS 60 Minutes will air a story, "A Not So Perfect Match" on familial searching of DNA databases. Familial searching, as I discussed last May on this blog, is the search of DNA databases for near misses that might be indicative of a family relationship between a known sample in a database and an unknown sample from a crime scene. This story is in part based on a paper of mine (along with collaborators Frederick Bieber and Charles Brenner) in Science that examined the feasibility of using kinship analysis to accurately identify family relationships in a very large database. Our conclusion: familial searching is quite practical to do, and would produce many new leads. The aggressive use of familial searching, however, raises a variety of policy, ethical and policy issues....

A broader point relevant to this blog are all of the data that are collected out there that are essentially relational, and how that information might be used. Further, to the extent that data have a relational dimension (i.e., information about me might provide insight about others I am connected to), individual-based protocols for consent might be problematic-- because third parties that might be affected are not asked for consent.

In any case, this story will be posted on the CBS News website in its entirety by Sunday evening in case you don't catch it when it airs.

Feel free to post reactions to the story here.
_________________________________________

Frederick Bieber, Charles Brenner, David Lazer, “Finding Criminals Through DNA of Their Relatives,” Published Online May 11, 2006, Science DOI: 10.1126/science.1122655, 2006.

March 29, 2007

Taking an idea where it shouldn't go...

One sign that the whole network idea may have gone a bit too far: when I was flipping around the channels last night was one with the header: “Jesus networked with people from varied backgrounds”.

March 15, 2007

The Venn diagram of "terrorist" and "network"

Alex's recent post on Dark Networks started me thinking about how and why network structures evolve. This turned into a conversation with my friend and colleague Jon Lindsey, a PhD candidate in Political Science at MIT and an intelligence officer in the Navy soon to deploy in Iraq. He suggested that the network form is actual quite fragile with respect to organizational forces: given the opportunity, many of these organizations will grow quite hierarchical with the standard bureaucracies we would expect to see else where.


Sure enough, from the Combating Terrorism Center at West Point comes a project called "Harmony and Disharmony: Exploiting Al-Qa'ida's Organizational Vulnerabilities". The full report (116 pp, pdf) makes for an interesting read, but I am fascinated by the translated original sources that are included. I was particularly drawn to the Al-Qaeda employment contract, with details as mundane as:


The bachelor Mujahed qualifies for a round trip ticket to his country after one year from
joining the organization. He can take a one month vacation. He doesn't get reimbursed if
the ticket is not used, but he has the right to change it to a ticket to perform
the pilgrimage. This period starts from the date of joining AL-QAEDA.

The married Mujahed and his family qualify for round trip tickets to their country of origin
after two years, and one month vacation. Tickets can not be reimbursed if unused.

This seems as bureaucratic as any employment agreement that I've signed, with the exception of the organizational goals. This document was discovered in Afghanistan, where the the administration of the organization appears to have become more mundane that the "4th Generation warfare" theory might suggest. Absent the constraints of a hostile, ubiquitous surveillance and law enforcement state, Al Qaeda starts to look a little like The Office.

At the same time, the demand for a network analysis approach for understanding and combating terrorism might be coming even more important. The Washington Post reports that


With new plots surfacing every month, police across Europe are arresting significant numbers of women, teenagers, white-skinned suspects and people baptized as Christians -- groups that in the past were considered among the least likely to embrace Islamic radicalism.

The demographics of those being arrested are so diverse that many European counterterrorism officials and analysts say they have given up trying to predict what sorts of people are most likely to become terrorists. Age, sex, ethnicity, education and economic status have become more and more irrelevant.

Absent the ability to use profiling to detect targets of interest, the type of data that the NSA was accused of collecting (albeit illegally) might be very useful for prevention and threat containment.

March 10, 2007

Dark Networks – The international network of the Red Army Fraction (RAF)

German researchers recently published an in depth analysis of the RAF, Germany’s terror group of the 1970s. While this topic has been studied well in the past, their work allows for deeper insights into the RAF’s international network. Apparently, the RAF was well connected to other European and Palestinian terrorist groups. They shared stolen weapons, received training in the Middle East or gained access to safe zones. For example, a hand grenade stolen by the RAF in a raid on a German based US Army depot was later used by Carlos, the Venezuelan terrorist. Cooperating Palestinian groups were even secretly supplied by the KGB officially endorsed by Breschnew, the former chairman of the communist party. Despite that, from ideological point of view RAF’s cooperation with the Palestinians was somewhat paradox as the RAF justified their actions with antifascism or members called for sympathy/support of the "left" with Israel in the Six-Day war.

Unfortunately, the two volumes called “Die RAF und der linke Terrorismus (The RAF and the left wing terrorism)” by Kraushaar (Ed.) are only available in German at the moment.

February 6, 2007

Strong ties are those that look for you…

See excerpts on New York Times story on disappearance of Microsoft researcher James Gray. This is a remarkable illustration of social capital—the social capital story underpinnings are: (a) that Gray had many ties; (b) these ties were to people who control important resources; (c) that these relationships were strong enough to mobilize these individuals; (d) that their network of relationships enabled effective collective action, and (e) the role that technology has played in allowing a bottom up, distributed, effort in the search.

On this last point, check out/ or even help look for James Gray, by going to the Amazon Mechanical Turk. AMT is a tool to allow distributed work on very large projects. From the website:

“During the last 5 days, Mechanical Turk workers looked at more than 560,000 images from 3 satellites, covering nearly 3,500 square miles of ocean. A group of experts is currently reviewing the images that workers identified, and sending their results to the appropriate authorities.”

My best wishes to Dr. Gray and, to his friends and family in their efforts to find him.

DL

____________________________________

To see full NYT article .

February 3, 2007
Silicon Valley’s High-Tech Hunt for Colleague
By KATIE HAFNER
SAN FRANCISCO, Feb. 2 — When James Gray failed to return home from a sailing trip on Sunday night, Silicon Valley’s best and brightest went out to help find him. After all, Dr. Gray, 63, a Microsoft researcher, is one of their own. The United States Coast Guard, which started a search Sunday night, suspended it on Thursday, after sending aircraft and boats to scour 132,000 square miles of ocean, stretching from the Channel Islands in Southern California to the Oregon border...

Continue reading "Strong ties are those that look for you…" »

December 28, 2006

Social Networking Comes to Healthcare

Dan Novak from IBM posted about an article from yesterday's Wall Street Journal on the SOCNET list. Since this is something I'm personally very interested in, I thought I'd share the posting with this community, with more to come from yours truly about SN in healthcare...

Quoting Dan Novak:
"Social networking to help patients and improve healthcare in today's Wall Street Journal (12/27/06).

My company has been supporting Patient Centric Networks (PCN), primarily from an Information and Communication Technology (ICT) point of view.

"Patient Social Networks" have the potential to educate, provide support, contribute to research, improve healthcare, and save lives.

Full article requires subscription, here is the free preview"

Continue reading "Social Networking Comes to Healthcare" »

December 8, 2006

What makes online ties sustainable?

Recently we heard more and more that online social networking platforms don’t really work - Alexa teaches us, that people tend to sign up for MySpace, Facebook or openBC, but platform providers have the hardest time to keep the network alive: people tend to sign up, but don’t or only infrequently come back to their profile.
This made my co-author Thomas Langenberg, EPFL Lausanne in Switzerland, and me start to think about the question: What makes online ties sustainable? We came up with a research design that looks at four different phases of a life cycle of online ties.

Here is the abstract of our paper:

Recently, the Pew Internet & American Life Project published a study about the number of social relations people maintain online and the omnipresent question was raised again: are actual face-toface contacts declining over time and are they replaced by online social interactions. Our virtual life is scattered in online profiles across sites such as openBC.com, Friendster.com, Match.com or MySpace.com. There are currently more than 400 different online social networking sites – with new sites popping up every day. Building on existing factors of persistence and sustainability of network ties in general, we address the key research questions: Which factors lead to the creation, maintenance, decay and reconnection of online network ties? Our research draws on prominent issues in the social network literature, which address the gap between research on offline and online social networks. We examine individual, dyadic, structural and content-related characteristics to understand how and why actors in different phases of their life cycle turn to online ties. Within the presented research framework, we derive propositions and develop a research design to collect and analyze qualitative and quantitative network data. The overall goal is to develop recommendations on how online social networks can become sustainable over time, and we develop questions and avenues for further research.

We came up with the following taxonomy of online vs. offline networks in our paper:

sntypology.jpg
You can download the full paper on our Working Paper website of the Program on Networked Governance.

Full citation:

Mergel, I./Langenberg, T. (2006): What makes online ties sustainable? A Research Design Proposal to Analyze Online Social Networks, PNG Working paper No. PNG06-002, Cambridge.

December 5, 2006

Open-Source Spying

Interesting piece in Sunday's NYT Magazine on the potential for collaborative web-type tools to help the US intelligence community to "connect the dots". The essential conundrum, as highlighted in earlier postings, is how to facilitate the critical information sharing within the community, and not to allow it to seap out in damaging ways. Not clear that there is a way to achieve both.

Open-Source Spying

By CLIVE THOMPSON
Published: December 3, 2006
(New York Times Magazine)

When Matthew Burton arrived at the Defense Intelligence Agency in January 2003, he was excited about getting to his computer. Burton, who was then 22, had long been interested in international relations: he had studied Russian politics and interned at the U.S. consulate in Ukraine, helping to speed refugee applications of politically persecuted Ukrainians. But he was also a big high-tech geek fluent in Web-page engineering, and he spent hours every day chatting online with friends and updating his own blog. When he was hired by the D.I.A., he told me recently, his mind boggled at the futuristic, secret spy technology he would get to play with: search engines that can read minds, he figured. Desktop video conferencing with colleagues around the world. If the everyday Internet was so awesome, just imagine how much better the spy tools would be.

But when he got to his cubicle, his high-tech dreams collapsed. “The reality,” he later wrote ruefully, “was a colossal letdown.” ...

December 4, 2006

Geography, demography, the emerging Republican majority (?), and social networks

A post script on the election: Over the last couple of decades there has been a lot written on the emerging “Republican majority.” The 2006 election, providing majorities to Democrats in both houses in Congress, seemingly belies this assertion. However, it is only one election, with results in part a referendum on an unpopular President, and in part a referendum on a poorly run Republican Congress. Neither changes the basic tectonics of American politics, so it is worth revisiting the Republican majority thesis.

In part, these assertions rest on geography (red states are generally growing in population faster than blue states), and in part on demography (red voters growing faster as share of electorate than blue voters).

The argument about geography rests on the fast growth rates of the (red) South and West, and the slow growth rates of (blue) Northeast. These divergent growth rates have to be driven by some combination of migration and divergent fertility/mortality. (These growth rates over the short run have to be primarily the former, although for interesting piece on the fertility component, which argues that conservatives have more children and thus will inherit the body politic. I think the argument in the final paragraph, however, trumps this fertility analysis). To the extent that the growth rates are driven by intra-US migration, the question is what happens when blue staters move into a red state. There are two possibilities: (1) the blue staters are socialized into the ways of their adopted state; or (2) the politics of their adopted state shifts incrementally toward a shade of purple.

I have not seen a definitive study of this issue (please post a comment if you have); however, the election of Webb (and Kaine in 2005) in Virginia offers an intriguing hint. Northern Virginia, in the orbit of Washington DC, is home to many transplanted blue staters, and is a nice laboratory to examine the social persuasion effects of a red state on these blue staters. The electoral results in 2005 and 2006, where Northern Virginia provided Webb and Kaine their margins of victory, suggests that the migrants to red states, rather than taking on reddish hues, may be coloring their adopted states blue.

Of course, Northern Virginia has some particular characteristics—e.g., many of its residents work in Washington DC, and thus may be predisposed to be more pro-government. Further, migrants to Virginia have clustered in Northern Virginia, rather than scattering randomly through Virginia, perhaps allowing them to escape the social pressures to conform. However, such clustering is typical; further, it is interesting to see that the Rocky Mountain West, long the bedrock of the Republican party, has become more competitive. This might well be in part because of migrants (some combination of Mexican immigrants, ex Californians, and East Coasters) have pulled these states left. It is also probably because libertarian, Goldwater, types are somewhat disenchanted with current big government Republicanism. (Note, New Hampshire, the state with the single biggest Republican to Democratic shift in 2006, also falls into this category, with substantial numbers of migrants from Massachusetts, but also with those libertarian tendencies.)

Stepping away from social networks for a moment, it does seem like the Bush years will prove a lasting liability for the Republican party. Party affiliation “sets” in early adulthood. The perceived success or failure of the sitting President thus gets recorded in the party affiliation of those individuals, with electoral effects for the next 50 years. Much as the Reagan-Bush I years produced a generation of Republican-tilting voters (or at least, neutrally balanced voters), the Clinton-Bush years may be producing a generation of Democratic-tilting voters. (my single favorite political graphic of this election season). I suspect that this graphic, if anything, understates the generational tilt toward Democrats, inasmuch as many of the earlier generation of Democrats are Southern white Democrats who tend vote nationally like Republicans. This demographic tilt toward the Democrats is also pushed further by the apparent drop in the Republican share of the Hispanic vote from 2004 to 2006. We will see if this is just temporary; however, conventional wisdom (correctly, I think) is that share of the Hispanic vote is critical in future elections because of its increasing size.

The bottom line, however, is that it is becoming increasingly difficult to assert that the axis of American politics is shifting toward Republicans. The election of 2006 is just another bit of evidence, among many.

November 6, 2006

Coming home on election day? Republicans, Democrats, and their partisan networks

There are some surveys suggesting a shift toward Republicans in the waning hours of the Congressional campaign. While one has to be careful about interpreting shifts, which are often stated in terms of a gap between the parties or candidates, because the gap is subject to twice the sampling error reported in a survey, i.e., +/- 4 becomes +/- 8, resulting in illusory swings, there is some scholarly basis for believing that Republicans would “come home.” In particular, following from work by Robert Huckfeldt and John Sprague and collaborators, discussions about politics likely are most active very close to the election. These discussions essentially activate the influence of ones milieu. Because partisans tend to have confidants who are like themselves, this tends to mean that wavering partisans will return to their partisan ways come election day. In this case, my understanding is that Republicans were wavering more than Democrats, and as a result a return to the fold would naturally create some shift to Republicans. Keys to watch, then, are (1) how independents vote (my understanding is that they are tilting heavily toward Democrats); (2) relative turnout of Democrats, Republicans, and Independents; and (3) the geography of these shifts—e.g., with respect to the House, a shift toward Republicans would not matter much in the South (except for a couple of seats), but a shift in the Northeast and Midwest would matter a lot. Regarding the first point, independents do tend to turn out at lower rates than partisans. Diana Mutz has done some fascinating work on the immobilizing effects of being exposed to cross-cutting pressures. An interesting question is whether, if Independents are tilting heavily toward the Democrats, they will be exposed to fewer cross-cutting pressures (as compared to if they were splitting down the middle), and thus likely turn out at higher than expected rates.

In a matter of hours, we should know...

November 3, 2006

The dark side to knowledge sharing: U.S. Web Archive Is Said to Reveal a Nuclear Primer

In an earlier entry I had blogged about an effort at "open source intelligence"-- a large archive of Iraqi documents that the US government placed on the web in the hope that the same type of open source dynamics underlying wikipedia, linux, etc, would fuel insights that the more old-fashioned, hierarchical US intelligence community could not match. The subtext was a hope by some on the Hill that evidence of imminent Iraqi nuclear capacity would be uncovered.

Well, chalk one up for clunky hierarchiy, because it turns out that hierarchical controls on information are sometimes more important than creative processing of that information.

A story in the NYT today (see excerpts below) discusses how that archive included sensitive information about how to build nuclear weapons that would shortcut the development process for other aspiring nuclear powers. In short, while the US was looking at the archive as an open source community for intelligence, Iran and others might look at it as an open source community on bulding nuclear weapons.

There is a much broader issue here about open source processes (which I am defining pretty broadly here), when they are desirable, when they are not. Essentially: in an adversarial system there is a need to prevent the adversary from participating in the knowledge sharing process. This, at times, can be hard or impossible to do. Another recent instantiation of this is the recent military effort to shut down "milblogs"-- blogs by soliders.

The tough tradeoff is that there are sometimes clear benefits to open source processes in terms of knowledge creation, but that that value creation process can destroy value if that knowledge falls in the wrong hands.

Continue reading "The dark side to knowledge sharing: U.S. Web Archive Is Said to Reveal a Nuclear Primer" »

August 17, 2006

NSA wiretapping ruled unconstitutional

Returning to the NSA thread for a moment, a judge in Detroit today ruled that the NSA wiretapping program is unconstitutional. Quoting from the Washington Post:

"U.S. District Judge Anna Diggs Taylor wrote in a strongly-worded 43-page opinion that the NSA wiretapping program violates privacy and free-speech rights and the constitutional separation of powers between the three branches of government. She also found that it violates a 1978 law set up to oversee clandestine surveillance."

Here is a copy of the decision:
http://www.washingtonpost.com/wp-srv/nation/documents/wiretap_ruling.pdf

August 8, 2006

Information Sharing and Institutional Impact

A very interesting piece in today’s Washington Post about information sharing in homeland security—excerpts below. I am also linking to the DHS IG report on the Homeland Security Information Network. The report and article do highlight (1) the social component to information sharing (but also the institutional impact on those social networks); and (2) the limitation of technology at this time in overcoming the key bottlenecks in necessary information sharing.

For full article click here

For report click here

In Arizona, Officials Share Data the Old-Fashioned Way
Wednesday, August 9, 2006; Page A07


On a recent drug bust in Phoenix, law enforcement officials found a vial of white powder that made them suspicious. They called Arizona's joint federal, state and local counterterrorism center, which dispatched a team to investigateThe powder turned out to be triacetone triperoxide (TATP), a highly sensitive explosive similar to the substance that failed "shoe bomber" Richard Reid was found to be carrying on a transatlantic flight in December 2001. Word of the discovery reached the FBI in Washington within 10 minutes. Although arrests and interrogations later determined there was no terrorist connection, it was just the kind of rapid information-sharing envisioned by intelligence changes implemented after the Sept. 11, 2001, attacks caught the intelligence community by surprise.

But the news did not travel through any of the sophisticated new high-speed communications systems built to facilitate rapid information flow between Washington and domestic counterterrorism's front lines. Instead, it went the old-fashioned way, with an Arizona police official walking across the hall to tell his friend in the local FBI counterterrorist task force, who then picked up the telephone and called headquarters.

According to a June report by the DHS inspector general, 2 percent of 9,500 registered users of the Homeland Security Information Network -- the department's two-way computer portal -- logged on to the system each day.
Although 360 state officials were cleared to use HSIN's separate secret portal, users averaged 27 a month.

For Beasley, who has retired from the state police and has been asked by Washington to help teach other state centers how to operate efficiently, success in Arizona began with his relationship with Churay. "When you talk about Washington and all those systems they're developing that are going to interconnect everybody in the country and everybody in the world, that's good," he said.

"But the reality is, on a day-to-day basis you have to go into those systems," he [Beasley, who designed the state center in AZ] added. "Most people, if they're operational, don't have the time. That's where that personal relationship, day to day, is absolutely critical. This business is built on trust."

May 26, 2006

NSA phone log analysis

Excerpts from USAToday article on May 22nd that give greater insight into the types of analyses that the NSA may be doing.
Pre-9/11 records help flag suspicious calling
Updated 5/22/2006 11:46 PM ET
By John Diamond and Leslie Cauley, USA TODAY

The intelligence officials offered new insight into one way the database of calls is used to track terrorism suspects.
The officials, two current U.S. intelligence officials familiar with the program and two former U.S. intelligence officials, agreed to talk on condition of anonymity. The White House and the NSA refused to discuss the template or the program.

Using computer programs, the NSA searches through the database looking for suspicious calling patterns, the officials say. Because of the size of the database, virtually all the analysis is done by computer.
Calls coming into the country from Pakistan, Afghanistan or the Middle East, for example, are flagged by NSA computers if they are followed by a flood of calls from the number that received the call to other U.S. numbers.
The spy agency then checks the numbers against databases of phone numbers linked to terrorism, the officials say. Those include numbers found during searches of computers or cellphones that belonged to terrorists.

May 11, 2006

Science article: Finding Criminals Through DNA of Their Relatives

On a related note, regarding the data mining of relational data, today with co-authors Frederick Bieber and Charles Brenner, I had a paper come out in Science on using DNA databases to identify relatives of those in the databases: "Finding Criminals Through DNA of Their Relatives." In the extended entry is the Harvard press release.

In a small nutshell, we use a combination of Monte Carlo simulations and genetic kinship analysis to demonstrate the feasibility of identifying the source of a crime scene sample if a close relative of the source is in one of the offender DNA databases maintained by various jurisdictions around the country (and the world). We find, for example, that in a database of 50,000, if one rank ordered the most likely close relatives to least likely close relatives, if a close relative were in the database, that relative would be ranked first about half of the time, and in the top 100 99% or so of the time. Accuracy could be improved further if additional genetic data were extracted from the samples. In short, kinship analysis would indirectly incorporate most of the first degree relatives of those in DNA databases (there are 3+m people in US databases). We discuss the policy and ethical implications of these findings, which I will not go into here (see release below).

There are two interesting things, from the perspective of this bog, that I would like to note. First, is the paradigm shift that kinship analysis represents. If one looks at the discourse around the expansion of DNA databases in this country, it has focused on the attributes of those going into the database-- convicts for certain crimes, what is the probability of recidivism, etc. But genetic data is inherently relational, and thus the technology is really incompatible with the discourse. The interesting thing--although, perhaps this paper will just drop into the bottomless well, never to be heard of again-- is what happens when credible research comes out highlighting the disjuncture.

Second, is the more general issue around data mining of relational data, individual choice, and privacy. As both this article and the NSA data mining highlight, so much information is inherently relational. The fact that I have certain characteristics may say something about people that I have various types of connections to. This does create certain conundrums for constructions of privacy that rely on individual choice, since with relational information, what I choose to reveal about myself reveals something about others-- i.e., there are informational externalities. In this day and age, this issue is endemic, and suggests that certain types of decisions about privacy must be necessarily communal and not individual in nature.

Continue reading "Science article: Finding Criminals Through DNA of Their Relatives" »

NSA data collections

In my earlier postings on the NSA data gathering program, I posed the question, what data would I want if I were the NSA, and all I cared about was potentially detecting terrorist activity, etc. My answer was simply, I would want them all. Further, I pointed out that the principles laid out by the administration did little to point to any line short of "all" that was desirable. Why stop with international calls? etc.

This was closer to the truth than I imagined, per USAToday story that broke today. Excerpts:

"It's the largest database ever assembled in the world," said one person, who, like the others who agreed to talk about the NSA's activities, declined to be identified by name or affiliation. The agency's goal is "to create a database of every call ever made" within the nation's borders, this person added.

In defending the previously disclosed program, Bush insisted that the NSA was focused exclusively on international calls. "In other words," Bush explained, "one end of the communication must be outside the United States."…
Sources, however, say that is not the case. With access to records of billions of domestic calls, the NSA has gained a secret window into the communications habits of millions of Americans.

The data are used for "social network analysis," the official said, meaning to study how terrorist networks contact each other and how they are tied together.

April 26, 2006

Who votes for whom? - Voting patterns in the US senate

I would like to draw your attention to an entry by Andrew Bond posted a few weeks ago in his blog "Analytical Visions".

senate_netw.jpg

Andy recently published a follow up on US senate voting patterns. One of the Program on Networked Governance research projects called "Connecting to Congress" is collecting a lot of data on how the Internet might transform Congress ways of connecting citizens to elected officials so we are always interested in that type of research. We will also use SNA with some parts of the data during the course of the project. Related to Andy's post is a paper by Wang/Mohanty/McCallum that draws on voting records from the US Senate and UN. Their SNA simultaneously discovers groups of entities and also clusters attributes of their relations, such that clustering in each dimension informs the other. In short, legislators many times cluster around a topic regardless of their party membership.

Finally, if you have ever wondered who supported who's bill in congress you should check out the embedded link.

March 28, 2006

Open source intelligence?

Excerpts from an interesting story in today's NYT, where Iraqi government documents are being placed on the web. What is most illuminating, really, is the discussion of the motivation of the bloggers, etc, who are examining the documents. The questions, in an exercise like this, are (1) why would anyone put substantial effort into something like this? and (2) do meritorious contributions "float to the top?

Re the first, it seems clear in this case, consistent with open source projects, that key motivations are the possibility of "making a difference" and of getting attention (e.g., getting on the front page of the NYT). Re the second, I suspect what will happen, in contrast to open source software projects (which have a certain degree of central management to guarantee continued coherence), is that multiple truths will emerge, where what is ideologically meritorious will float to the top in different communities.

_____________________________________________________________

March 28, 2006
Iraqi Documents Are Put on Web, and Search Is On
By SCOTT SHANE

WASHINGTON, March 27 — American intelligence agencies and presidential commissions long ago concluded that Saddam Hussein had no unconventional weapons and no substantive ties to Al Qaeda before the 2003 invasion.

But now, an unusual experiment in public access is giving anyone with a computer a chance to play intelligence analyst and second-guess the government.

Continue reading "Open source intelligence?" »

March 21, 2006

Social Networks and the Business World

Social Network Theory and its principles are applied by more and more companies in a way that some of us might not be aware of yet. So what we buy, how we rate products/services, post in forums, pictures we upload or present of ourselves on the web is significantly influencing other, likeminded individuals. In return we are influenced by the network cluster we belong to for a specific habit and the like. Collaborative filtering is a key component of using social networks for different purposes. Further information can be found here. Below you will find a list of various industry and application examples:

Social Networking plattforms
There are the obvious social networking online plattforms. Among them are the open business and personal contact manegement oriented like Tribe.net , openbc, friendster or the inivitation only communities like asmallworld. Either planned or already implemented users can take advantage of added services (search functionality, messaging) by paying a monthly fee 10< USD. Furthermore, there are the rather dating/partner match making plattforms like match or eharmony.

Retail/eCommerce
Most of today's ecommerce sites use collaborative filtering to improve sales, cross-,up- and downselling. A prominent example are Amazon's recommendations based on various user behaviours on their website.

Music/Radio
Tapping into our musical tastes Last FM, Genielab or Pandora present us with streaming music. Here the main business model lies in linking to the respective ecommerce sites like Apple's iTunes.

Books
The same applies to the area of what we might want to read next which also serves ecommerce purposes.

Movies and more
MovieLens is a free service provided by GroupLens Research at the University of Minnesota. Whether, you want to book a hotel, whole vacation there are numerous examples of collaborative filtering apps on websites.

Pictures
The most prominent example for sharing, managing and searching for pictures is Flickr or myspace. The latter gaining revenues from online-ads.

Search engines
As I have elaborated in an earlier entry on google bombs the network structure (ties) play an important role in search engine algorithms.

Knowledge Base and OpenSource
The online encyclopedia Wikipedia builds on the power of decentralized, voluntary collaboration building an enourmous depository of multi-language information. Whether it was the development of Linux, Mozilla/Firefox or MySQL all rely on and consist of social networks. Further examples of openSource projects can be found at Sourceforge.

SNA Consulting
As we can see the character and concepts of networks is mainly utilized for recommendations. Actual applications of SNA is done by a few companies and consultants like Rob Cross, IBM, Orgnet or Visiblepath. These companies try to uncover the informal networks within organisations to improve knowledge sharing, initiate change or bridging silos.

Finally, you can always follow latest trends in social network analysis at PNG's subpage on SNA by Ines Mergel.

March 14, 2006

NYT magazine article: Can Network Theory Thwart Terrorsts?

There was a piece in the NYT Magazine on Sunday on the surveillance program, titled “Can Network Theory Thwart Terrorists?? by Patrick Radden Keefe

It offers some basic ideas in applying network analysis to detecting terrorists, although it contains, I believe, a couple of mistakes. First, it states that “Network academics caution that the field is still in its infancy and should not be regarded as a panacea.? Not a panacea, certainly true; but in its infancy? Certainly not—the modern field can trace its roots to the 1930s with the work of Moreno, Newcomb, and others. However, this is really just a quibble, in that the analysis of very large full network data (e.g., see Nathan Eagle’s posting on his work on the phone data of an entire country) is in its infancy, due to the lack of tools and data until recently.

A more serious apparent error is in the following statement:

“The use of such network-based analysis may explain the administration’s decision, shortly after 9/11, to circumvent the Foreign Surveillance Court. The court grants warrants on a case-by-case basis, authorizing comprehensive surveillance of specific individuals. The NSA program, which enjoys backdoor access to America’s major communication switches, appears to do just the opposite: the surveillance is typically much less intrusive than what a FISA warrant would permit, but it involves vast numbers of people.
In some ways, this is much less alarming than old-fashioned wiretapping. A computer that monitors the metadata of your phone calls and e-mail to see if you talk to terrorists will learn less about you than government agent listening in to the words you speak.?

From what we have seen in the news reporting (excerpted in previous postings), however, the NSA program involves more than analysis of metadata. It also involves recording and content analysis through computer algorithms of the conversations of large numbers of individuals, with snippets selected out for human analysis. Indeed, I believe this was the whole basis of the controversy—the stories on the metadata came after the wiretapping story. While I wouldn’t say that metadata pose a non-intrusion, this is a much bigger deal. Indeed, as I believe I have said in previous postings, I would consider a digital recording of my conversation, content coded, to be a greater intrusion on my privacy than having two FBI agents listen to my conversations, with paper transcripts going into a file somewhere. Those bits do not decay like human memories, or easily lost, like files stored in the bowels of an enormous bureaucracy.

February 26, 2006

More on NSA network analysis....

More on NSA: A piece in yesterday’s NYT provides more insight in the data analysis capacity of NSA. It confirms that the NSA is likely doing a combination of social network analysis with voice recognition to select out promising snippets for further (human) examination.

Some excerpts:

… databases [are] maintained at an AT&T data center in Kansas, which now contain electronic records of 1.92 trillion telephone calls, going back decades.

A former AT&T official who had detailed knowledge of the call-record database said the Daytona system takes great care to make certain that anyone using the database, whether AT&T employee or law enforcement official with a subpoena—sees only information he or she is authorized to see, and that an audit trail keeps track of all users. Such information is frequently used to build models of suspects’ social networks.
The official… said every telephone call generated a record: number called, time of call, duration of call, billing category, and other details…. [N]ames, addresses, credit card numbers are in a linked database….

The National Security Agency has invested billions in computerized tools for monitoring phone calls around the world—not only logging them, but also determining content—and more recently in trying to design digital vacuum cleaners to sweep up information from the Internet.

An earlier NSA patent, in 1999, focused on a software solution for generating a list of topics from computer-generated text. Such a capacity hints at the ability to extract the content of telephone conversations automatically. That might permit the agency to mine millions of phone conversations and then select a handful for human inspection.

In 2003, Virage [a company]… began supplying a voice transcriptions product that recognized and logged the text of television programming for government and commercial customers. Under perfect conditions, the system could be 95% accurate in capturing spoken text. Such technology has potential applications in monitoring phone conversations as well.

From: “Taking Snooping Further: Government Looks at Ways to Mine Databases,? by John Markoff (B1), NYT, Feb 25.

February 5, 2006

Contributions and Comments to our Blog: Some Clarifications

Dear Community of the Complexity and Social Network Blog,

Thank you very much to all of you who have tried to raise and drive this blog with your contributions and comments. We very much appreciate your efforts. It is important for us to build a strong community to make this blog an effective forum for discussions on "Complexity and Social Networks".

However, we feel that there is a need to send out a short message regarding the "rules and regulations" for the blog. In the past days we received quite a few comments on our blog entries regarding the latest developments in Europe and the Middle East (Click here for the link), which we unfortunately were not able to publish. We found the comments clearly politically and ideollogically motivated and did not refer to the blog's underlying theme (Complexity and Social Networks).

Our goal with this blog is NOT to facilitate discussions about more or less political issues. In our blog we want to disucss academic theories of "complexity and social networks", their application in the real world, and related research methodologies. In our latest blog entry for instance, we tried to apply an academic framework in a real life setting. In doing so, we attempted to facilitate a scholarly discussion on the effects of network structure on (organizational/institutioal) influence. Therefore, our blog does not provide a forum for discussion of political or religious issues. We think there are many other and probably much better places out there (on the internet) to do that.

Along these lines, we would like to thank you once again for your contributions, but please be aware of the fact that we cannot publish your posts when they become either too "political" or are not linked to the overall focus of this blog.

Regards,

Your PNG Blogging Team

February 3, 2006

Resisting network pressure or how to regain stability - Denmark vs. the Muslim Community

In the following entry, we attempt to become a little more substantial.

In the early phase of the current "Mohammed-Cartoon" episode, the discourse took place between the Danish publisher and the local Muslim community. However, now the news have spread and it has become a global debate. Yesterday, the UN (Kofi Annan) made a statement, individuals were threatend or taken hostage and governments are fearing economic and socio-political consequences (such as Denmark or France).

By taking a network perspective, we thus raise the following (research) question: How does a country's (structural) position in a global network of public and private sector organizations, NGO's, and a wide variety of civil society organizations impact its scope of political and economic action?

In 1997, Rowly published an article in the Academy of Management Review proposing a structural classification of stakeholder influences and potential organizational reactions. His conceptual framework looked as follows:

rowley.JPG

Continue reading "Resisting network pressure or how to regain stability - Denmark vs. the Muslim Community" »

February 1, 2006

Danish/ EU newspapers vs. the Islamic World - An Example of Network Failure?

12 Cartoons (Example 1, Example 2) posted in a Danish newspaper last September (10/30/2005) and thereafter caused a series of aftershocks which are felt now - 4 months later. Although the newspaper received a bomb threat right after the day of publishing and money was awarded for the execution of the cartoonist the real uprage just started this week with reports and reprints of the Mohammed cartoons all over Europe.

While we don't want to join the discussion on freedom of speech, making jokes about one's own God and the like, we would rather take a look at the network aspect of it. Although 11 amabassadors of islamic countries publicly protested about this and 5000 muslims marched through the streets of Copenhagen it took 3 months until this was fully recognized in the Arabic and international community. Why did it take so long, although media reported and all types of ICT's were utilized?

Let us take an "information diffusion" and "social network" perspective. One might argue that today social networks are so dense that information spreads quickly by any means (internet, blogs, wom etc), especially when the topic is potentially critical. According to our theories we would have expected that Arabic communities are very well connected so that news from Denmark rapidly diffused throughout the Islamic world. Also, we would have thought that ambassadors act as "information hubs" in their societies, and thus reactions would have occured much earlier.

So what happend to the "news" within the last 3 months? Did it take so long until a critical mass was built to react? Shall we assume that, based on powerlaw, it took 3 months to reach a consent within the strongest 20% of the Islamic community? Along these lines this would mean that 20% of the latter community gives us the feeling that the other 80% think simillarly. Is it really like that, or what might be other explanations for our observation?

International reports:
Wikipedia (very detailed)
CNN
BBC
Bloomberg
Brussels Journal
Die Welt blog (English)
Der Spiegel (German)

Arab Positions:
Arab News

This post is a collaborative effort with Thomas Langenberg.

January 30, 2006

Retrospective vs prospective data analysis

Before I leave the NSA thread, one more thought about the utility of the data mining phone log data, recording phone calls, etc. It is important to note that pattern recognition is only as powerful as the ancillary intelligence that the government would have to complement the phone data. Obviously, in the absence of any indications of who is high risk, the signal in the data is infinitesmal.

A corollary to this is that pattern recognition of today's data will be much more effective in the future-- because the amount of complementary information will only go up. We may have no reason to pay attention to Joe, except that he was 3 degrees removed from someone in some terrorist's phonebook. However, in the future we may receive solid intelligence that Joe is a terrorist. At that point, it would be a pretty good idea to go through all of the data you collected about Joe-- if you have it sitting in a hard drive somewhere. If you haven't been recording, and collecting information from the switches, etc, it may be impossible to reconstruct afterwards. That is, "retrospective" pattern analysis is likely to be much more effective than prospective pattern analysis. Of course, this in turn points to a strategy of collecting data now and asking questions later, which again brings us back to the issues around privacy and collateral usage in aces.

January 23, 2006

Privacy issues and possible collateral uses of NSA data

Just closing the loop on the NSA data collection for now. Having discussed the potential utility of the data for investigation of terrorists, I now turn to the potential collateral use of these data. The potential is considerable—probably a lot greater than for combating terrorism— especially if the data are retained.

First of all, the data could be used for other criminal investigations. In fact, it is difficult to draw a logical line separating terrorist activity from other criminal acts—e.g., many more people are harmed by other types of crime each year in the US than by terrorism. Obviously, there is a potential for terrorist acts that are especially devastating, and terrorism has particular political significance. But from an empirical/utilitarian calculus, it is difficult to justify why different tools are appropriate for preventing deaths from terrorist activities and not for other criminal activities.

In fact, one would anticipate that for people who “live on the grid? in the US, the data produced could be very powerful for investigations. Consider simply the potential use of locational information from cellular phones, where one might be able to identify who was near the crime scene at the time of the crime. Even if the perpetrator of the crime was sensible enough to turn off their cell phone (and many would not), this would be an effective way to identify material witnesses.

Continue reading "Privacy issues and possible collateral uses of NSA data" »

January 17, 2006

Excerpts from Spy Agency Data After Sept. 11 Led F.B.I. to Dead Ends

Excerpts from today's front page NYT article below. Three things to take away from this article: (1) that there was a snowball-type of analysis from the relational data collected; (2) there was some degree of content analysis of communication (although not clear that voice recognition was involved); and (3) the signal to noise ratio seems to be unacceptable (to the point of being humorous).

The next entry on this (and then I will return to more traditional points of discussion for the blog) will be on potential
collateral uses of these data.

January 17, 2006
Spy Agency Data After Sept. 11 Led F.B.I. to Dead Ends
By LOWELL BERGMAN, ERIC LICHTBLAU, SCOTT SHANE and DON VAN NATTA Jr.

In the anxious months after the Sept. 11 attacks, the National Security Agency began sending a steady stream of telephone numbers, e-mail addresses and names to the F.B.I. in search of terrorists. The stream soon became a flood, requiring hundreds of agents to check out thousands of tips a month....

Continue reading "Excerpts from Spy Agency Data After Sept. 11 Led F.B.I. to Dead Ends" »

January 10, 2006

ABC/Washington Post survey on privacy

A bit of a digression from social network analysis, but following from the recent items on the NSA, some results from Washington Post/ABC survey on surveillance/domestic spying.

7. In investigating terrorism, do you think federal agencies are or are not intruding on some Americans' privacy rights?

Are Are not No opinion
1/8/06 64 32 4

Continue reading "ABC/Washington Post survey on privacy" »

January 9, 2006

NSA data mining—what patterns to look for: expansive scenario (II)

A more expansive scenario would be that the NSA collects all phone log data from US sources as well as non-US calls that pass through US switches, plus locational information from cell phones where available (+ e-mail traffic, etc).

The expansive scenario offers a significant security and logistical advantages to the NSA. The security advantage is that under the more limited scenario, the NSA would have to share critical security information with telecomms, by asking them for information about only certain individuals. That delimited information is terribly sensitive intelligence—by telling telecomms who they want to monitor, etc, it is essentially telling them who the government has received intelligence about.

Continue reading "NSA data mining—what patterns to look for: expansive scenario (II)" »

January 7, 2006

NSA data mining—what patterns to look for (I)

So, what data mining could one do with the data the NSA has collected from telecomm companies? Obviously, it is still unclear as to what is being collected, so this is quite speculative, which is a little different from my normal role of cautious academic. My hope is that this speculation, in the end, will yield some productive discourse about this important subject. I also want to make clear that I am not endorsing (or condemning) such data mining for now. Later I will discuss some of the privacy and policy issues. For now, I just want to do a thought experiment of how one might analyze these data in a fashion that might detect terrorist activity.

My assumption here is that the objective is to identify candidate nodes (individuals) for surveillance.

I am going to start with what I consider a less expansive scenario. In this particular scenario, one is starting out with some phone numbers and e-mails that are designated as “high risk?—e.g., from other intelligence. A simple analysis would simply snowball outwards from these high risk nodes to their contacts, and to their contacts’ contacts, etc. As one snowballs outwards, one will likely find overlaps, where some nodes are members of multiple circles. In the simplest analysis, the more circles that a node is a member of (and the closer to the center of those circles), the higher risk they should be considered.

Obviously, the analysis should get substantially hairier than that, because of the nature of the sampling from the network. For example, I am guessing that the identifications of high risk nodes are not independent events. Imagine that an Al Qaeda cell is identified and its members apprehended in Jordan, and their computers, address books (or equivalents) acquired. One would then snowball outwards from these contacts. However, to find overlap among the contacts of these cell members presumably conveys different information than if one found overlap among the contacts of different cells from different countries (presumably the latter would be more significant).

One could devise a weighting system that depends on the number of paths that go through a particular node, other information about nodes, etc, to develop a ranking of who should be watched. These weights could be validated by fitting them to part of the network data, and then examining whether the technique was effective at identifying those nodes that you knew were already “high risk.?

Ideally, one would use communication data going back in time as far as possible—thus, while telecomm companies are sharing data, you would want them to go back as far as possible. This would also be useful in case you wanted to do sequence and timing analysis—e.g., it’s not just who you call, but it’s when you call (say after some event), or that you called Anne after Joe called you.

Obviously, there are lots of difficult issues re sampling. Further, one would hypothesize that any terrorist worth their salt would be careful about recording contact information, and, more generally, their use of electronic communication. And I would guess that most of the people that terrorists communicate with are non-terrorists, and their contacts, in turn, are even less likely to be terrorists, so the vast majority of people caught in this net are going to be non-terrorists. So, to mix metaphors, one may have removed from the haystack proportionally more hay than needles, but you are still left with a very large haystack with just a few needles.

Once one has identified some risky nodes, the next step would be to monitor actual communications. Presumably, the NSA has finite capacity to have humans listen to conversations, and thus the key management question is how to allocate this scarce resource. The first level of monitoring would therefore simply be recording of conversations. Presumably, this is fairly cheap to do, so, putting civil liberties concerns aside, one would adopt a pretty low risk threshold for recording. This would allow going back in time for human monitoring if an individual were subsequently identified as high risk. A second level, if it is technically possible (at some level it surely is), would be to apply voice recognition to those recordings, where the content of conversations would adjust the evaluated risk level of those nodes. Further, such voice recognition could pick out candidate snippets of conversations for human monitoring. Such “snippet-based? monitoring, I think, would explain why the FISA court process was circumvented, since it might result in the brief, human-based monitoring of a very large number of people (conceivably exceeding the number of warrants approved by the FISA court in its history very quickly), and in the computerized monitoring of a still larger numbers of people. That is, the oversight process specified by FISA would be unable to cope with the sheer volume of requests. Further, the basis of monitoring these snippets is probably weaker than what has traditionally been brought before the FISA court. It would also explain why some defenders of the policy (who presumably know more than has been publicly released) have stated that having a computer monitor your conversation was not a privacy intrusion (thus suggesting that a major component of the program did involve computerized monitoring).

This is the less expansive scenario that I have come up with (although how expansive it is depends on a number of parameters—how many steps out one goes from the initial sample, what is the threshold for monitoring, etc, so the actual numbers of people who are in some fashion caught in the net might number anywhere from thousands to millions). This is a pretty rudimentary analysis, as compared to how one would actually do it, but I think has the essential ingredients. My next entry will consider a more expansive scenario.

January 4, 2006

NSA data

Before I get to what might be done with the data, a little more on the data that has been collected, from James Risen’s book, State of War: The Secret History of the CIA and the Bush Administration (2006):

Following President’s Bush’s order, US intelligence officials secretly arranged with top officials of major telecommunications switches carrying the bulk of America’s phone calls. The NSA also gained access to the vast majority of American e-mail traffic that flows through the US telecommunications system. (p. 48)

The telephone network today is digital and computerized, but is still built around a switching system that routes calls from city to city, or country to country, as efficiently and quickly as possible…. In addition to handling telephone calls from, say, Los Angeles to New York, the switches also act as gateways into and out of the United States for international communications…. [I]t is now difficult to tell where the domestic telephone system ends and the international network begins. (pp. 49-50)

One of the secrets of the Internet is that its infrastructure is dominated by the United States, and that much of the world’s e-mail traffic, at one time or another, flows through telecommunications networks that are physically on American soil. (p. 51)


December 30, 2005

Social network analysis, the NSA, and “pattern analysis?

The story about the NSA eavesdropping program has received a lot of attention over the last week. The follow up story has received somewhat less attention, but may be more important, see story from December 24 NYT: Spy Agency Mined Vast Data Trove, Officials Report (by ERIC LICHTBLAU and JAMES RISEN):

“What has not been publicly acknowledged is that N.S.A. technicians, besides actually eavesdropping on specific conversations, have combed through large volumes of phone and Internet traffic in search of patterns that might point to terrorism suspects. Some officials describe the program as a large data-mining operation.?

“A former technology manager at a major telecommunications company said that since the Sept. 11 attacks, the leading companies in the industry have been storing information on calling patterns and giving it to the federal government to aid in tracking possible terrorists.

"All that data is mined with the cooperation of the government and shared with them, and since 9/11, there's been much more active involvement in that area," said the former manager…

This is a remarkable story, and raises some interesting questions: (1) exactly what data are telecomm companies sharing with the government; (2) what could usefully be gleaned from these data; and (3) what are the privacy implications?

There is a lot more we don’t know about this story than we do know, but it is worth beginning a discussion on the value and the costs of these data under different scenarios of exactly what information is being shared. My next few entries will aim to begin a discussion on these issues, grounded primarily in a social network perspective.

Briefly, what data do telecomm companies have? Focusing on the telephone data, for now, they have (1) phone log data; (2) varying amounts of locational information for cell phones; and (3) varying amounts of information linking individuals to particular phone numbers (e.g., not so much for some pay as you go phones, more for other types of phones). My understanding is that little remains of the bits that flow over the network (i.e., the content).

These are thus a type of social network data, along the lines of my preceding entry on the “behavioral flows? of relationships. That is, for any given dyad one can observe the timing and duration of calls.

Whose phone data is being tracked? It is not clear from the article. Clearly, the focus is on international communication (domestic to international, and international to international calls routed through switches that are on US soil). Is purely domestic communication also being tracked? The article suggests not:

“This so-called "pattern analysis" on calls within the United States would, in many circumstances, require a court warrant if the government wanted to trace who calls whom.?

This sentence is ambiguous, however—e.g., given that the sharing of data by the telecomm companies is voluntary, what are the statutory limits on their sharing data with the government? Is there a prohibition on a telecomm company voluntarily handing over information to the government regarding one of their customers’ phone logs? I do not know of such a prohibition, but if a reader does, please do comment.

For the next entry: Given that these are essentially social network data, from what we know from the research on social networks, what insights might they yield?