Complexity and Social Networks Blog: Social network analysis, the NSA, and “pattern analysis�?

August 2008
Sun	Mon	Tue	Wed	Thu	Fri	Sat
					1	2
3	4	5	6	7	8	9
10	11	12	13	14	15	16
17	18	19	20	21	22	23
24	25	26	27	28	29	30
31

« Virtual Stock Markets - Proving the Powerlaw? | Main | NSA data »

30 December 2005

Social network analysis, the NSA, and “pattern analysis�?

The story about the NSA eavesdropping program has received a lot of attention over the last week. The follow up story has received somewhat less attention, but may be more important, see story from December 24 NYT: Spy Agency Mined Vast Data Trove, Officials Report (by ERIC LICHTBLAU and JAMES RISEN):

“What has not been publicly acknowledged is that N.S.A. technicians, besides actually eavesdropping on specific conversations, have combed through large volumes of phone and Internet traffic in search of patterns that might point to terrorism suspects. Some officials describe the program as a large data-mining operation.�?

“A former technology manager at a major telecommunications company said that since the Sept. 11 attacks, the leading companies in the industry have been storing information on calling patterns and giving it to the federal government to aid in tracking possible terrorists.

"All that data is mined with the cooperation of the government and shared with them, and since 9/11, there's been much more active involvement in that area," said the former manager…

This is a remarkable story, and raises some interesting questions: (1) exactly what data are telecomm companies sharing with the government; (2) what could usefully be gleaned from these data; and (3) what are the privacy implications?

There is a lot more we don’t know about this story than we do know, but it is worth beginning a discussion on the value and the costs of these data under different scenarios of exactly what information is being shared. My next few entries will aim to begin a discussion on these issues, grounded primarily in a social network perspective.

Briefly, what data do telecomm companies have? Focusing on the telephone data, for now, they have (1) phone log data; (2) varying amounts of locational information for cell phones; and (3) varying amounts of information linking individuals to particular phone numbers (e.g., not so much for some pay as you go phones, more for other types of phones). My understanding is that little remains of the bits that flow over the network (i.e., the content).

These are thus a type of social network data, along the lines of my preceding entry on the “behavioral flows�? of relationships. That is, for any given dyad one can observe the timing and duration of calls.

Whose phone data is being tracked? It is not clear from the article. Clearly, the focus is on international communication (domestic to international, and international to international calls routed through switches that are on US soil). Is purely domestic communication also being tracked? The article suggests not:

“This so-called "pattern analysis" on calls within the United States would, in many circumstances, require a court warrant if the government wanted to trace who calls whom.�?

This sentence is ambiguous, however—e.g., given that the sharing of data by the telecomm companies is voluntary, what are the statutory limits on their sharing data with the government? Is there a prohibition on a telecomm company voluntarily handing over information to the government regarding one of their customers’ phone logs? I do not know of such a prohibition, but if a reader does, please do comment.

For the next entry: Given that these are essentially social network data, from what we know from the research on social networks, what insights might they yield?

Posted by David Lazer at December 30, 2005 11:19 PM

Comments

Similar developments are currently underway in the European Union. In late December 2005 the EU passed new regulation for storing telephone and internet data of telecommunication providers for 6-24 months. However, only connection and not content data should be stored by vendors. Slowly but steadily we are moving towards the transparent citizen...

Posted by: Alexander Schellong at January 2, 2006 6:56 AM

David, yesterday's Washington Post had an article by Walter Pincus that addressed your questions on the type of information made available:

www.msnbc.msn.com/id/10667276

Of course, it will likely be some time before we can say we have fully reliable information about the extent and intensity of the spying program.

Posted by: Ken at January 2, 2006 1:06 PM

Thanks Ken. Here's the title of the story and some relevant excerpts...

NSA gave other agencies surveillance data
Information from wiretapping was processed, cross-checked

By Walter Pincus
The Washington Post
Updated: 7:01 a.m. ET Jan. 1, 2006

Today's NSA intercepts yield two broad categories of information, said a former administration official familiar with the program: "content," which would include transcripts of a phone call or e-mail, and "non-content," which would be records showing, for example, who in the United States was called by, or was calling, a number in another country thought to have a connection to a terrorist group. At the same time, NSA tries to limit identifying the names of Americans involved.

...

The NSA would sometimes monitor telephones, e-mails or fax communications in cases where individuals in the United States -- and sometimes people they contacted -- were linked to an alleged foreign terrorist group, officials have said. The NSA, officials said, limited its decisions to follow-up with more electronic surveillance on an individual to those cases where there was some apparent link to terrorist sources.

But other agencies, one former official said, have used phone numbers or other records obtained from NSA in combination with wide-ranging databases to look for links and associations. "What data sets are included is a policy decision [made by individual agencies] when they involve other than terrorist links," he said.

Posted by: David Lazer at January 2, 2006 11:26 PM

Complexity and Social Networks Blog

August 2008

4

Editor Login

Convener in chief:

Editors:

Categories

Archives

Recent Entries

Recent Comments

Notification

Social network analysis, the NSA, and “pattern analysis�?

Comments