Is feedback the new Tower of Babel?

May 30th, 2010 by Tommaso De Benetti

In a globalized world, more connected by the minute, an insight into the minds of people virtually anywhere on the planet is only a click away. Researchers have access to incredible amounts of information about consumers and their preferences. Age old mysteries like whether people prefer Sonic the Hedgehog or Knuckles the Echidna can finally be answered.

Yet this does not necessarily guarantee a wide or representative range of opinion. Like everything else in life, when it comes to data, quality is more valuable than quantity. Companies that overlook this, or do not commit sufficient resources to gathering an accurate sample, are likely to miss out on major opportunities.

One of the problems is that it is difficult to access ideas and points of view outside a particular community. Many of my fellow Italians, for example, might be quite surprised to hear what the rest of the world is saying about their popular leader. Equally, how other cultures are tackling the same issues is often overlooked, as is how products are perceived across different continents. Differing life philosophies and cultural norms also affect interpretation of facts in a way which is not always obvious.

Broken English
Last year I had the chance to fly to Malmö, Sweden, and take part to the Nordic Game Conference. Together with a friend I did some networking, attended as many seminars as possible, and interviewed people for a games podcast we are producing. Amongst many interesting seminars, Insomniac’s Community Director Ray Schneider delivered a challenging lecture on how, as a game developer, you should always maintain a dialogue with the various communities that orbit your games. This applies to online multiplayer games, but also to any game likely to build a fan base.

During the speech he stressed how Insomniac monitors player feedback on games, and places great emphasis on it during the pre-production phase of sequels and new IPs. This feedback comes in two forms: automatically generated statistics (why one weapon is more popular than another, how many players got stuck in a specific level, why everyone tends to choose the same four characters from a list of thirty), and the opinions or gut feelings of users and specialized press. Gathering and understanding this information is hugely valuable for product designers.

Everything was humming along well until I asked Ray how he dealt with feedback from people who do not speak English. For a moment, before answering me, I had the feeling Ray was slightly embarrassed. He then replied, not completely confidently “we rely on our community for help.”

The problem for Insomniac, like many other producers of high quality games, is that its games are played by a global audience which is only readily defined in a nominal sense. My educated guess, based on their latest high profile IPs (Ratchet and Clank and Resistance), would be that their target audience is mostly middle-class male teenagers and young adults (14-28 years old).

Even within such a narrow group, the abundance of cultural differences between individuals makes the priority given to English feedback hazardous. Although these opinions are more accessible for developers, they represent a small fraction of a much bigger cake.

If a game featured, say, a scantily-clad lead female character, reactions to her design could differ significantly between US, Japan and Europe on the basis of many factors (age, education, etc.). While some non-native English speaking players could express their opinion on official community forums, it is unlikely that their comments would give a good indication of how the game has been received in the area they come from. Most players would instead be discussing it in forums and news groups accessible only to people who speak their specific language.

My sentiments exactly
Ideally, developers would assess every piece of feedback, regardless of what language it is in. But this is simply unrealistic for teams whose main preoccupation is trying to nail down the next triple A title.

This is where sentiment analysis comes into play: what if all that feedback could be crowdsourced to native speakers, better able to understand both the nuances of the language and the cultural framework from which the opinions originate? The untapped potential for exceptionally precise targeting is enormous.

Developers, like everybody else, often do not know what they do not know. With the wider feedback that crowdsourcing would allow, products could be better targeted at established and developing markets around the world.

Believe it or not, it’s my guess that many such people may just prefer games that don’t involve hedgehogs at all.

---

Share

 Print article

---