Page 28 of 43« First...1020262728293040...Last »

Crowd Labs Incorporated

November 22nd, 2010 by

microtask_crowd_labs.jpgThe history of experimenting on humans doesn’t have what you’d call a spotless reputation. Google it, and you get Nazis, CIA mind control, and conspiracy theories about Guantanamo Bay and Ritalin. And that’s just in the first ten hits.

Given its unpleasant past, crowdsourced workers might, at first, be less than enthusiastic about their growing popularity as subjects for human research. Thankfully, academia has moved on since the bad old days of the Stanford prison experiment and the worst thing most modern participants ever have to deal with is a badly worded questionnaire.

A researcher’s paradise?
Social scientists, psychologists and economists constantly require thousands of people to take part in experiments. Trouble is, running a lab is expensive: assistants to pay, participants to hunt down and schedules to organize. Compare that (as Lauren Schmidt did at CrowdConf this year) to crowdsourced labor: a global pool of potential subjects who’ll work for a fraction of the price, and don’t need travel expenses.

It’s a concept the research community is just beginning to get behind. Some classic thought experiments – like the prisoner’s dilemma and the Asian disease problem have already been tried out using Amazon’s Mechanical Turk.

One ingenious Harvard PhD candidate – John Horton – also used Mechanical Turk for several “field tests”, where workers undertook tasks without knowing they were in an experiment. So while people thought they were merely tagging a few photos, they had, in fact, unwittingly contributed to a paper on worker productivity. (For now we’re ignoring the obvious ethical issues regarding disclosure.)

Another group put together an entire iPhone app that gets you to record how often you daydream throughout the day (apparently some people even recorded while they were engaging in more active bedroom activity, which is taking dedication to science a bit too far, in my humble opinion). Over 2000 people joined the study, making it the largest in its field.

Troubled Minds
Not everyone though, is convinced about the wisdom of crowd-based research. We are talking about academics here after all – they’re basically professional skeptics.

One big issue is how you know if participants are really who they say they are? Mechanical Turk lets you select workers for age, gender, location etc, but only based on what workers themselves claim. Was that task really completed by a housewife in Minnesota, or was it a kid in Mumbai on his mom’s laptop?

There’s also worry over whether low-paid, crowdsourced workers will give enough attention to experiments. I imagine it’s pretty infuriating to have your research data skewed because a subject was checking out Beyonce videos in between answers.

As crowdsourcing becomes a more accepted research technique, people will no doubt find ways around these problems. One solution could be a dedicated site or API (a kind of virtual lab) which allowed researchers to customize experiments easily, and attracted workers who knew, at least roughly, what to expect.

Whatever the future holds, it looks like our knowledge of who we are, what we do and why we do it is set to get a serious boost from the crowd.

If you’ve already been experimenting with the crowd, as always, we would love to hear from you.


Crowds against the Machine: will digital workers soon be digitized?

November 18th, 2010 by

01 :: Robot :: 01 by Denise Cortez @ FlickrUntil October this year, I would have put driverless cars in the same category as jetpacks, robotic maids and space elevators. All the stuff, in other words, that a time traveler from the 1950’s would expect to see gathering dust in our garage (“Gee Mister, the Internet sure is a blast, but when do we go visit the moon base?”).

Now, thanks to Sebastian Thrun & co at Google, we’re apparently one step closer to the world of tomorrow. During the summer Google’s self-driven cars secretly clocked up over 140,000 miles around California. The cars are modified versions of Toyota’s Prius, perhaps chosen as a nod to the Japanese who produced the first driverless car back in 1977.

All seeing eyes
The Google cars “see” using a combination of swivel camera, laser and radar sensors (which makes the human eye seem even more impressive). The vast amount of visual data is processed, along with detailed maps, by the car’s Artificial Intelligence system which uses it to navigate the road.

There’s been a lot of fanfare from Google about improved road safety and impact on climate change, but no one’s really sure what they plan to do next with the cars. The cost of the project hasn’t been released, but I’m guessing it’s a little more than your average Prius production run.

This isn’t the first time Google have experimented with cutting edge A.I. –designing Google Street View, they used sophisticated computer vision systems called ConvNets to blot out faces and license plates. ConvNets are capable of “deep learning”– they modify their own parameters when fed new information (something many football fans seem to struggle with).

An artificial struggle
The sophistication of this technology is awesome, but is it good news for the crowdsourcing industry? Without ConvNets, wouldn’t tagging Streetview images be exactly the kind of microtask you’d expect outfits like ours to distribute to digital workers? If a bunch of robots will soon be able to complete such tasks faster and cheaper, will microtasks become just a footnote in the history of A.I.?

Before we all pack up and donate our office space to cybernetics, it’s worth considering the fundamental premise of distributed microtasks. As most people who read this blog know, microtasks generally fit into a category known as human intelligence tasks, or HITs. They exist because humans are still far better than some things – especially related to vision and language – than computers.

HITs – while often very simple for us – are designed for the nuanced, responsive intelligence that only humans possess. Although your Dad’s dusty Commodore 64 can multiply numbers faster than Dustin Hoffman in Rain Man (unless the answer is $100), the latest supercomputers are, overall, still laughably rudimentary compared with the human brain.

With our 3 billion year evolutionary head start, it is barely surprising that robots are nowhere near as sophisticated as we are. (Of course, a chicken/egg issue arises here, and a smarmy robot might argue that if we were smarter we could make them smarter again. At that point I suggest unplugging it.)

The fact is that there may never be a time when A.I. meets all of our peculiar demands. The vast cloud of crowdsourced labor can react, almost instantly, to a mind-boggling variety of tasks – from rating the emotional impact of an image, to decoding a 300 year old sentence of text. New applications and companies are constantly popping up – from real time mapping aids for the blind to the latest crowd funding initiatives (see The Daily Crowdsource if you want to try and keep pace – you might even spot a familiar face).

In fact, the more computers enter everyday life, the more microtasks there are likely to be for distributed workers to help with. So while the machines may be rising, I’m betting it will be a while before they rival the power and ingenuity of the two billion people online right now.


Adventures in Primetime

November 15th, 2010 by

microtask_riemann_zetaIn high school, math was my personal bane. Like countless others, I found trigonometry tedious and calculus incomprehensible. In time though, as I became older and (ahem) more mature, I started to cultivate a serious fascination for hard science. My conversion happened when I learned of the crazy characters and mad schemes behind so many groundbreaking discoveries. I might not be able to grasp every detail of quantum physics or number theory, but the writer in me can’t resist a good story.

A few years ago, I had the chance to read two amazing books: Fermat’s Last Theorem and The Music of Primes.

Before you give up and click over to Amazon to get the latest Dan Brown thriller, let me tell you just one incredible fact. If it hadn’t been for the work of mathematicians Pierre de Fermat (1601-1665) and Bernhard Riemann (1826-1866) you wouldn’t be able to buy anything on Amazon (or anywhere else online) at all.

The indivisible truth
To make a long story short, both these men were all about primes. Fermat tried to find a way of testing for prime numbers, while Riemann attempted to predict the distribution of primes from zero to infinity. Primes (for those who were skipping classes) are numbers that can be divided only by 1 or themselves. They have no apparent logical order: 2, 3, 5, 7, 11, 13, 17… the list goes on forever.

So what’s so important about a bunch of random numbers?

Well, primes are the basis of all other numbers or, as Marcus du Sautoy elegantly puts it, the atoms of arithmetic. Over the centuries, mathematicians have been obsessed with finding bigger and bigger primes. In 1996, distributed computing joined the search with GIMPS, a project that allows users to run free, prime-hunting software on their PCs. There are thousands of GIMPS volunteers who together have donated teraflops of processing power.

The very randomness of primes has also proved useful, helping to forge the algorithm that now protects all our credit card numbers and online transactions.

Decoding the future
In 1859, Riemann finally realized that, working in a multi-dimensional space, a logic in the distribution of prime numbers could actually be found (illustrated here by XKCD). Unfortunately, Riemann never produced a full proof of his theory – legend has it his cleaning lady accidentally burned the papers – so the Riemann hypothesis remains unsolved.

Even in the absence of concrete proof, many people have adopted the Riemann hypothesis as a working model. Take the RSA algorithm, which is now used to encrypt all transactions with electronic money (including Amazon’s 32 sales per second). Without going into details, the principles of RSA encryption are built on the insights of Fermat and Riemann, but decryption is only possible using the unproven Riemann hypothesis.

A million dollar question
Since it was formulated, there have been two attempts to crowdsource a solution to the Riemann hypothesis. In 1902 David Hilbert included it in his list of 23 great math problems, inviting anyone who could to provide solutions. A century later, the Clay Institute established seven millennium prize problems, offering $1,000,000 for each correct solution.

So far, I regret to say, crowdsourcing has not brought forth the solution to the world’s most complex mathematical problem. Who ever does find the answer – whether it’s an individual, a group, or even (who knows) a huge, widely distributed, network of collaborators – I’m sure the breakthrough will come with another crazy, unbelievable story. And, given the importance of prime numbers, no doubt security services, corporations and governments will be following developments pretty closely as well.


Science fun: the protein shake that makes you a genius

November 11th, 2010 by

microtask_foldit_proteinFor the past few nights, my life has contained far more protein than normal. I’m not talking about the variety best cooked medium-rare on a barbecue, but the brain-food found in the science-meets-crowdsourcing game Foldit.

You got to know when to hold it; know when to foldit
Foldit is a free, online protein-folding game. Players tweak, shake and wiggle chains of amino acids into stable 3D shapes (think biochemical origami).

The usefulness of proteins is explained in detail on the Foldit site. Suffice it to say, if we want to cure Alzheimer’s, AIDS or even just allergies, we’re definitely going to need them.

For years computers have struggled with the pattern recognition and problem solving ability needed to fold proteins. Foldit’s creators found that people – even those with no scientific background – excelled at these skills. In a study published in Nature, human players were pitted against the latest modeling software, called Rosetta. Out of ten puzzles the humans won five, drew three and lost two.

The game has been widely reported as a “man beats machine” story, but the real genius of Foldit is that it allows man and machine to work together. Once Rosetta has created the initial shape of a protein, people take over – using their superior strategy and risk-taking ability to find the optimum “fold”.

World of Foldit
Other crowdsourced science projects exist, such as Galaxy Zoo, which has over 60 million contributions. What makes Foldit stand out (as if the chance to fold protein molecules wasn’t thrilling enough on its own) is that it is effectively a massively multiplayer online science game.

Now, I’d be the first to admit Foldit is no World of Warcraft. But although the interface and gameplay are pretty basic, Foldit does use the power of game mechanics to keep you hooked. There are tutorials, points, rewards and levels – all of which give a sense of progression and increasing status. It’s easy to imagine a future version where you play with your own, lab-coat wearing Avatar (who’d no doubt start off as a humble, virtual teacher’s aid and work up to a tenured professor).

There’s also an active and growing community of some 60,000 Foldit players. Teams and soloists compete for top rankings, chat on forums, write wikis and can enter contests with cash prizes. This “metagame” activity creates a great buzz around the game, as well as motivating players and encouraging competition and innovation.

What’s in a game?
Over and over again people have shown they are prepared to invest huge amounts of time, energy and skill into playing online games. A commitment level many employers would envy. In their book Total Engagement Byron Reeves and Leighton Read envisage a future workforce demanding the same level of satisfaction from their job as they get playing games.

Foldit, while perhaps a little primitive, uses a game to harness the power of the crowd in aid of science. A first step perhaps in a much wider trend (one that online, crowdsourced work is ideally placed to exploit) that could ultimately blur the distinction between work and play.


Now that’s what I call a remix: confessions of a YouTube-oholic

November 8th, 2010 by

C-mon & Kypski More is Less videoYouTube can be a dangerous place. A place where your time, and occasionally your sanity, vanish without trace. Trouble is, it’s also the best place to find out what crazy, creative stuff the “crowd” is up to. What I bring you today, after many hours of dangerous research, is some of its latest work. Hope you enjoy it as much as I do!

Shining happy people
Previously on this blog we explored fan-made, “sweded movies” (essentially parodies of big, Hollywood productions). Believe it or not, there is an even more bizarre cult of cinema crowdsourcing: the mood-changing remix. Here fans get busy cutting, pasting and changing the soundtrack in order to transform trailers – or even whole movies. You can now experience The Shining re-cut as a feel good comedy, The Big Lewboski as a story of moving friendship, or (at the risk of losing precious childhood memories) watch Mary Poppins turned into a hellspawn witch.

Forty-one milliseconds of fame
Music is another example of a field where experimentation with widesourcing (a clever new word CrowdCloud dreamed up) never seems to stop.

Leading the way is the band C-mon & Kypski who’ve put together a crowdsourced video. In ‘One Frame of Fame’ each person is asked to copy the pose of a band member and take a picture (it’s indie music, so strictly no air guitar). The website uses software that runs on all browsers and, after rendering each shot, updates almost in real time. If you’ve always dreamed of leaving a mark on Earth for future generations, this is your chance (the video is already nominated for the UK Music Awards).

Innovative artists and hobbyists – who treat their followers as a resource, not an enemy – don’t just crowdsource, they source a new crowd. C-mon & Kypski have over 25,000 participants in their video. That’s 25,000 people who’ll want everyone they know to download the track. Fans also have deep pockets: Public Enemy have just announced they’ll be recording an entirely crowd funded album, after raising around $60,000 through Sellaband.

With a little help from their fans
In sharp contrast to the music industry (whose legendarily conservative attitudes I’ve been wincing over in the book Appetite for Self-Destruction, The Spectacular Crash of the Record Industry in the Digital Age) the gaming world seems eager to let the fans do the work. Acclaimed LittleBigPlanet for PS3 has over 3 million user created levels, with a sequel due out that allows even greater customization. Meanwhile converts to the retro sandbox game Minecraft are so keen they’ve made fan-based trailers to help spread the word – that’s before the game is officially out of development.

In the past, fans have famously tried to fight change (like the audience booing the night Bob Dylan went electric). Now they’re pushing it forward, showing artists how to get the most from new technology. The creative potential of the YouTube crowd is clearly immense, even if their projects are still a little rough around the edges. Left unhindered, people are often able to rethink and remix material in ways that would astonish (in a good way, well, mostly in a good way) the original creators.

So, I invite music execs, film producers and game designers to join me and spend a few hours surfing the YouTube crowd. It could just turn out to be the most valuable time they’ve ever wasted.


Page 28 of 43« First...1020262728293040...Last »