Watching Big Brother

August 10th, 2010 by

The summer of 2010 will be remembered for searing temperatures and soaring government debt. With more whistle-blowing than the World Cup final, it was also the summer of Wikileaks, the whistle-blowing internet site, and its weird, white-haired front man Julian Assange.

Thanks to Wikileaks, I now know that even if I am dying of thirst in a heat wave, I am better off avoiding Helsinki’s ATMs to draw cash for a drink. When I am kept awake at night by the heat and ATM fraud, I can put myself to sleep reading 92,000 secret documents it has published relating to the war in Afghanistan.

Opening the flood gates
But even all the attention Wikileaks is receiving, historians looking back at this time may see it as just the exciting edge of a much wider trend. For when I finish reading the files about Afghanistan sometime in 2020, there are now vast databases of government information to get stuck into. Unlike Wikileaks, these online databases are not leaked documents, but rather information voluntarily released by governments around the world. Although less sensational than Julian’s masterpiece, this information may prove to be even more useful.

Pressure on governments to publish public information has been mounting for some time. Reflecting this mood, in 2005 a non-binding European Commission directive encouraged increasing openness by member states. But it seems like it is this year that things have started to get moving. On his first day in office, President Obama instructed the heads of federal agencies to release as much information as possible. This has led to a surge in information available for organizations like Public.Resource.Org. Similarly, in the UK, the Government has recently launched the massive COINS database, giving anyone who cares to look an incredibly detailed look at UK Government expenditure.

A wolf in Google’s clothing
All this is great, but just releasing this information does not necessarily change much. The problem is that the information available on any one of these sites is so vast, that even if I never slept another wink, I would never get through it. It means in practice that the information is almost as impenetrable as it always was.

Happily, to coincide with the emergence of these databases are systems which allow the public to collectively and individually scrutinize this data in a meaningful way.

One method, relying on the collective efforts of many concerned citizens, is to crowdsource the analysis. So, for example, when the MP expense scandal in the UK broke, rather than have teams of reporters sifting through almost half a million expense claims, the Guardian newspaper crowdsourced the scrutiny of them, with some interesting results. Broken down into bite sized fragments and shared amongst many thousands of people, analyzing government is more a microtask than a massive undertaking.

But a crowd is not always necessary. New systems, such as Wolfram Alpha, allow even individual citizens to scrutinize large amounts of data in a meaningful way. Rather than searching data like Google, Wolfram Alpha computes it, applying code from Mathematica to its own vast library of curated data (which makes it more like Wikipedia than Google, in a way). In theory, at least, this will make it much easier to analyze huge data sets, such as COINS.

Government by the people, for the people
All this access to information obviously makes it much easier for the public to keep an eye on what their elected (or unelected) officials are up to. It also has profound implications for how government works.

Rather than fear the public scrutiny, many in government are using it to come up with better ways to manage services. One of the first things the US federal government CIO did after taking office was to create an online dashboard, detailing the government’s $70b technology spending. Along with allowing extra scrutiny of what they were doing, the aim is to get the public to offer suggestions on how to do things better and more efficiently. It is not surprising then that cash-strapped governments like California’s are implementing similar programs.

Just how much Wikileaks and COINS type databases, Wolfram Alpha and crowdsourcing will change the way we are governed is hard to say. Transparency and scrutiny of government will always be a good thing, but there are perhaps practical limits to how much power can be devolved to the people. If California’s experiment with direct democracy is anything to go by, sometimes it is better to let elected politicians get on and do their jobs. As with much of what we discuss in this blog, only time will tell just what the future holds.