Crowded House

A little over ten years ago, the team running SETI - the Search for Extra-Terrestrial Intelligence - found a groundbreaking solution to an enormous problem. The team's efforts to secure data feeds from radio telescopes had been immensely successful - too successful, in fact, to the extent that SETI was inundated with vast amounts of radio data which it had no way to analyse. The organisation was running on a minimal budget, and the kind of computing power required to search that much data for potential signals would have been too expensive even for a much better funded effort.

The solution reached by the SETI team at the University of California, Berkeley, was SETI@home - a distributed computing effort which sent out packets of radio data over the Internet to volunteers who gave up the unused power of their home computers to the processing efforts. The result was that SETI was able to tap into hundreds of teraFLOPS of processing power (stop sniggering, that's how it's measured) at minimal cost - buying a supercomputer capable of that would have bankrupted the organisation.

SETI@home wasn't the first distributed computing effort of this type, nor is it the most successful, but it was the first to really capture the imagination of Internet users. As such, it's not a bad place to stick a flag in the dirt and say, "here was born the crowdsourcing idea".

Crowdsourcing has come a long way since then, and the concept is still difficult for many people. It's easy to understand how the computing power of tens of thousands of normal PCs can be combined to outperform a supercomputer using some clever software. What's not quite so easy to understand is the sheer power which can be brought to bear on an enormous range of problems by applying the same principles of distribution not to computers, but to the people sitting at them.

"Two pairs of eyes are better than one, people will say when they're offered help in searching for something - but what about 100,000 pairs of eyes?"

Two pairs of eyes are better than one, people will say when they're offered help in searching for something - but what about 100,000 pairs of eyes? The reCAPTCHA service asks web users logging into websites to identify two words taken from scanned documents, which computers have failed to recognise. The input of thousands of users generates startlingly accurate results. When Wikileaks released thousands of documents last year, the Guardian engaged internet volunteers in combing the documents for crucial information - a task that would have taken the paper's journalists months if not years, and was completely beyond the comprehension of even the most advanced computer software, but could be accomplished through crowdsourcing in hours.

These are simple examples. The extent to which crowdsourcing has impacted the world we live in is extraordinary, and growing. Each of us in the developed world walks around with an Internet connection, a surplus of computer power and a high resolution camera in our pockets. The leveraging of that power using the paradigms created for distributed computing has the potential to overturn entire industries - perhaps even to topple governments.

What I want to talk about more specifically, though, isn't toppling governments - but it might be something even more profound. It's a logical progression from crowdsourcing which, somehow, almost nobody seems to have foreseen - and whose eventual impact, I'd argue, still isn't fully understood.