Winner takes it all?

August 17th, 2010 by Tommaso De Benetti

On this forum we spend a lot of time discussing the enormous potential of crowdsourcing, and how it is going to change all our lives for the better. At Microtask, we just love crowdsourcing. With that in mind, what I am about to say may shock you. I urge you to sit down.

Recently I read a rather disturbing article on crowdsourcing. The article focused on 99designs, a community for crowdsourced web pages and logo designs.

In theory it’s great. Say you want a new identity for your company. All you do is post a design brief on the website, and then sit back and watch the submissions roll in. At the end of the process you only pay for your favorite submission.

99designs has all the mechanisms you would expect to ensure chosen work is paid for, avoiding both copyright theft and exploitation of the community purely as a source of inspiration. Even so, there is much criticism of how it and similar sites (such as reDesignMe, MycroBurst, CrowdSpring) operate.

Minority report
Some claim that this kind of crowdsourcing incentivizes quantity over quality (note 99designs’ mantra “a new design uploaded on the site every 7 seconds”). There may be some truth to such claims, although presumably the fact that people are actively using the site suggests that they believe the results are worth the cost and effort.

People also argue that this “competitive crowdsourcing” exploits the contributors. As a user who commented on the article I mentioned earlier asked: “I want to know how many of you would show up at the office Monday morning if you had to compete with 92 co-workers for a single pay check at the end of the day?”. Note that the two criticisms are interrelated, in a way, given that the ability to only pay for one solution drives the cost down. (Of course an economist might argue that rational contributors factor in their chances of success as well as the possible reward when they make their decision to participate).

The idea that such communities lead to exploitation gathers weight when one considers that minors are prevalent amongst the contributors. Although 99designs is actively trying to stop people under 18 from accessing the community, their efforts are easily circumvented over the internet.

To this criticism one user replied “No one is forcing people to participate by sending in designs. This is all voluntary work. (…) For some of those 11-year-old designers, perhaps they really are showing an interest in art or design, and this gives them an avenue to exercise their nascent talents, or learn some designing skills, or software tools. Heck, I think it would be a great exercise for a school project in an Art/Design class to have every student work up a submittal as part of a class assignment.” (Without getting into this particular argument, personally I think that there is no doubt that a system like this would be a great teaching aid in certain circumstances).

Creative microtasks?
Leaving these debates aside, for me the underlying question remains: when a high quality, creative output is required, is competitive crowdsourcing the best solution? To my mind, the key problem with these crowdsourcing communities is that submissions are mutually exclusive. If one is selected all the others are automatically discarded or must be reworked and adapted for different projects. Many of the issues surrounding exploitation of contributors flows on from this fundamental problem.

Happily, there is another way. Crowdsourcing concepts like Microtask’s, which rely on a very low level of worker input, have an entirely different approach. Every submission (or “microtask”), when added to all the other small fragments, cumulatively contributes to the successful completion of the whole task. Everyone who completes a task to the standard required will get the reward they expect.

Such a crowdsourcing solution clearly has enormous potential for tasks involving recurring steps or well defined mechanisms, but its ability to deliver creative solutions has yet to be proven.

This is an area we are currently researching. We would love to hear your opinion on how creativity and crowdsourcing could coexist in the same sentence without sacrificing output (and life) quality. Don’t worry; all interesting opinions will be rewarded with equal amounts of our gratitude and respect.

Share Print article

Pingback: Tweets that mention Winner takes it all? -- Topsy.com
Jarkko

I have ran couple of design projects at CrowdSpring so I know a little about the topic.

You don’t compete with “92 designers” as if each design was submitted by a different designer, because many of those designs are simple iterations of a single design. As an example a logo project I ran at CrowdSpring got 161 entries submitted by total of 36 designers. Many of the reiterations were just very simple changes to the design, e.g. color variations, font changes, etc. which probably took designer only couple of minutes to make. In the end I was very happy with the result. Then I was also involved in another logo project ran in the “traditional way” by contracting the design to a single designer. The result of the design was quite poor in my opinion and it was scrapped by the company at a later date, and cost many times more to have it designed.

You don’t either “post a design brief on the website, and then sit back and watch the submissions roll in”. It’s actually quite active process and requires project holder to prepare a lot of feedback and guide designers to the right direction to get the results you want without limiting their creativity too much. Sure there are project managers who don’t give feedback, but that’s a lose-lose situation for everyone. For properly ran project there’s a lot of feedback going back and forth. Also some designers get into iteration loop, and some just choose to drop the ball if their initial design isn’t good enough.

I also tried to contact several designer for another design job in the traditional way, asking for quotations for the job. However, there is absolutely no guarantee that the designer you would get involved with is actually able to do good job either. All those quotations were also only a fraction of the price I ended up spending on the CrowdSpring project, but it wasn’t as much the matter of price as it was the matter mitigating risks and getting exactly the design I wanted. For the results CrowdSpring has 100% satisfaction quaranteed (IIRC, 99designs too), i.e. if you are not happy with any of the designs, you get your money back (minus the nominal submission fee) no questions asked. I haven’t had to use that option though, so I can’t comment if it’s as straight forward as it sounds though.

And finally, this process is actually great way to find a talent you can outsource work at later with an affordable price. This is what IIRC CrowdSpring also mentions in their FAQ. Many designers get further contracts directly from the project holders, and for example if I need a logo or other design project I have had before, I know now who to contact first.

So, I don’t know how “unethical” these crowdsourcing sites are as some people claim. I mean they can be an opportunity for talented designers to break into the industry and start their own design business.
http://www.parliamodivideogiochi.it Tommaso De Benetti

Jarkko, thank you to sharing your first hand experience. Anyone else?
Pingback: Kilpailu – uusi tapa ostaa ohjelmistokehitystä? – 67 %