Complexity and Social Networks Blog

« MyFairElection | Main | Silent doors and the electoral college »

30 October 2008

On data growth and growing concerns

Are you in favour of more efficient and effective government? Of course you are. If one counted the reasons given most often for any type of government reform, these two would score the highest marks.

It is widely recognised that the characteristics of information and communication technologies (ICT) have strong impact on both. Government was thus among the first to utilise ICT. In the early days, punch-card machines were used for the census, and electronic databases replaced large amounts of data stored in non-digital form (for example, in files) throughout government once the technology was available.

Because information drawn from data is at the core of everything government does - analysis, decision-making or verifying eligibility for access to public services, to name just a few - the proliferation of databases, data mining and ICT in general is unsurprising. However, it is this increase in databases, the kind of data being gathered, the way that data is protected (or rather the opposite) and the way it is used internally and externally, that has come under increased scrutiny and been criticised by many civil libertarians.

But the criticisms are not just about civil liberties. When governments implement ICT, outcomes vary. Large-scale projects such as the FBI's Virtual Case File, the UK's C-NOMIS or Germany's FISKUS either failed completely or largely exceeded their estimated budgets, wasting billions of taxpayers' money. There are, of course, also successful projects but, by and large, the expected impact of eGovernment in moving into a brave new world of efficient, effective and citizen-focused government administration has not happened.

The power of ICT
ICT has characteristics that need to be understood before carrying out any impact assessment. These characteristics underline why digital data and databases will continue to grow in the future, and why it is necessary to find balanced governance mechanisms for ICT, for the organisations they are embedded in, and for us - the individuals using them.

ICT allows information processing, coordination and flows to be structured without the common boundaries of roles, organisational relationships and operating procedures found in government. As a consequence, the relationship between information and the physical factors of organisational size, distance, time and costs are altered.

Digital information makes geographical dispersion irrelevant, allowing for new forms of collaboration and networks. Information technologies facilitate the speed of communication and more selectively control access to, and participation in, information exchange.

Interestingly, the standardisation, routinisation and formalisation of information sharing are not only technical requirements for shared databases to be effective; they are also typical traits of bureaucracies.

Organisational memory that was once hidden in non-digital forms or an individual's memory can be stored, managed and analysed in digital form to improve knowledge or facilitate decision-making - helped by the fact that information storage, provision and search costs are virtually zero once information is digitised. Moreover, the human constraints of processing large quantities of information are reduced (for example, through the use of search engines), and software applications make it possible to combine and reconfigure data so as to provide new information.

This has been spurred by the rise of Web 2.0 applications such as social networking sites, mash-ups, tagging, and wikis, with the underlying philosophy that comes with it - i.e. mass collaboration and data sharing - further facilitating the growth of data.

The public has followed this trend on a scale that no one imagined. Younger people in particular store and share data about their activities, location, buying behaviour or personal lives like no other generation before, and periodical incidents of security breaches, identity theft and fraud have not reversed this trend.

Often, this behaviour is based on a conscious decision: millions of users joined corporate loyalty programmes (offered by, for example, airlines, hotels or shops) in return for personalised services, rebates or points that can be used in various ways. People may also just be following an intrinsic desire to share and connect. Wikipedia is one of the prominent examples of the powerful force of collaborative peer production.

Data is also gathered and stored by companies in ways which customers are unaware of, but while the public has less control over the activities of companies, there is generally greater concern when government is engaging in these types of activities.

The rise of government databases
The counter argument is that governments do not gather more data; they are just gathering and combining data in new ways (for example, databases, biometrics, face-recognition software, remotely readable chips (RFID)).

They do so for good reasons: national security, accountability, to provide better public services and to bridge organisational silos. Yet, since 9/11, more data is being sought indiscriminately rather than selectively, meaning that innocent people's data is included through law-enforcement agencies' screening processes.

Indeed, studies have shown that bigger DNA databases produce better results. This may argue in favour of creating a comprehensive DNA database containing information on all citizens and not just those convicted of crimes, as this may actually help to exclude suspects, save investigative resources and have a deterrent effect overall.

The automatic transfer of data about passengers flying from Europe to the US sheds light on another important aspect of the discussion on databases and data sharing. In a globalised world, should countries grant access to their domestic databases and how can they protect personal data beyond their national borders?

Incidents such as the day in November 2007 when the UK government managed to lose two CDs with unencrypted data of more than 25 million citizens underline four key issues relating to government databases.

First, the government has a mandate to protect the public's data. Second, data security is not only about technology. Thirdly, government needs strategies to manage digital trust. Finally, the characteristics that make ICT so valuable (for example, the ease with which it can be transferred) mean there are increasing vulnerabilities and risks: that data will be shared when it should not be, or that it will be lost, stolen or misused. At the same time, there is a risk that data will not be available when it should be.

Paradoxically, calls for new government databases and better interoperability do arise when the system of government fails. Accordingly, new databases to track and monitor individuals and institutions, or links between formerly separate databases, are built.

Moreover, many ideas for creating pro-active, multi-channel, one-stop and joined-up government simply do not work without databases. The volume of data to be collected will grow constantly in the near future as more government transactions are digitised, and cases of data being cross-referenced ('mined') will also grow as the relevant software improves.

Even if a government body decides to discard data, it faces many difficulties.

First, because data storage costs are continuously decreasing, governmental organisations prefer to keep everything, creating 'data cemeteries'. The expansion in the volume and kinds of data maintained by agencies have made it almost impossible to maintain an inventory of resources.

Second, interim systems sometimes bridge the incompatibilities between the old and the new system, thus keeping the legacy system alive and increasing its overall complexity. For example, the US' Internal Revenue Service launched a new software application to support a total quality management initiative, but never shut it down after the initiative ended. The amount of work it would take to resolve issues relating to data exchange with other systems were considered too high.

Moreover, these 'electronic mounds' accumulate massive quantities of rules that conflict with changes to other systems. For example, to control user access, user behavior or make sure different software applications can work together. The possibilities of storing and searching electronic information may also justify the development of large sets of these rules, so ICT does not always cut red tape. This is why some have proposed a combination of laws and technology to require and make it easier for data to be deleted - and thus "revive our society's capacity to forget".

Policy options
The expansion of databases puts greater burdens on the political-administrative-ethical calculus to strike the right balance between innovation and regulatory regimes. The following questions should be considered by policy-makers in the planning stages of initiatives that include setting up databases:

• What type of data should be collected and why?
• Who collects, maintains and owns the data?
• How is the data collected?
• How long should the data be stored?
• Should the data be shared and why?
• Who will be affected by making data more widely available?
• What impact will the data have on different stakeholder groups?
• Is the data aggregated?
• Is there a minimum opt-out provision for those who provide the data directly or indirectly?
• What security measures and policies are in place to protect the data?
• How to can the data be accessed and changed by those who provide it directly or indirectly?
• How can accuracy of the data be ensured?
• How can the data be reviewed and disclosed?
• Do third parties who provide or use the data have the same security standards and privacy policies?

Policy-makers should also consider educating the public better on issues such as privacy and identity self-management - a process which may need to begin as early as in elementary school. They also need to understand how trust and the perception of security in digital government is created.

In any case, there will be many alternatives for government, businesses and the public to choose from when incorporating ICT into their lives. The perception of what is right and wrong will evolve alongside the values they are measured against, and the databases and techniques they are applied to.

(a longer version will appear in the European Policy Centre's "Challenge Europe - Is Big Brother watching you - and who is watching Big Brother" publication)

References:
D. Lazer (Ed) (2004) DNA and the Criminal Justice System, Cambridge, MA: MIT Press.

V. Mayer-Schoenberger (2007) 'Useful Void: The Art of Forgetting in the Age of Ubiquitous Computing', RWP07-22, Faculty Research Working Paper Series, Harvard University, Cambridge, MA: John F. Kennedy School of Government.

National Research Council (2008) Protecting Individual Privacy in the Struggle Against Terrorists: A Framework for Program Assessment, Washington, D.C: National Academies Press

A. Schellong (2008) Citizen Relationship Management, Brussels: Peter Lang Publishing.

D. Tappscott; A. D. Williams (2007) Wikinomics, New York: Portfolio Hardcover.

The Economist 'Data Mining, 27 September 2008.

The Economist 'Privacy in Britain, 28 January, 2008.

The Economist Big, bigger, biggest', 28 February 2008

The Economist 'Identity parade, 14 February 2008.

Posted by Alexander Schellong at October 30, 2008 6:24 PM

.
• Business (3)
.
• Citizen Relationship Management (17)
.
• Commentary (7)
.
• Complexity (2)
.
• Current Trends (47)
.
• eGovernment (21)
.
• Events/Announcements (74)
.
• Innovation (14)
.
• Internet (18)
.
• Knowledge Sharing (20)
.
• Methodology (32)
.
• Networked Governance (18)
.
• Networks in political science (5)
.
• PNG Working Paper Series (6)
.
• Powerlaws (4)
.
• Reviews (2)
.
• Simulations (1)
.
• Small group networks (3)
.
• Small Worlds (2)
.
• SN in the news (72)
.
• Social Capital (3)
.
• Social Computing (12)
.
• Social Finance (1)
.
• Social Networking Platforms (11)
.
• Social Psychology (2)
.
• Social Software (25)
.
• Societal Networks (40)
.
• Technology (35)