The Pitfalls of Ratings-Driven Communities

Posted by Chris G. on May 23, 2017, 8:00:10 AM

3 minute read

We need to separate user-reviews from critics-reviews

Ratings Communities Gaming Reviews

The digital marketplace has become a river of reviews. Scores, stars, thumbs up and down drive the vast majority of digital purchases, empowering the average consumer like never before.

Dissatisfied with a business?

Leave a public display of frustration on social media and a rep will likely contact you within 48 hours to fix it with a fast remedy. Mobile apps constantly fish for positive reviews, not because they want your opinion, but to catch more stars than the competition, for the sole purpose of boosting their page rank higher.

Even mobile games have jumped on board, offering digital incentives for leaving reviews, without specifically requiring them to be positive (which would violate the market’s TOS).

Reviews aren’t going anywhere, anytime soon. So how do you get attention when the big dogs are competing for the top spots*?

* Mind you, the “top spots” are the hundreds, if not thousands of games in modern digital marketplaces. The sheer amount of content available is staggering, even when you take audience size into account. Last year the iOS marketplace was seeing more than 500 new games a day, not taking other applications into consideration.

Beware the Bots

Well, you could fight dirty. The most common tactic?: Autonomous bot accounts that give positive reviews to your game while leaving overwhelmingly negative feedback to your competitors. Yes, it’s dirty and dishonest, and we don’t recommend it. But despite all the techie roadblocks implemented to eliminate (or at least suppress) this strategy, people will always find a way to work around them. Watch the news? Then you know, even US politics can’t escape the bots, which is arguably more difficult than getting your game a ratings bump.

At this point you may be thinking, chill, everyone! Why does this matter so much? Don’t better games get the attention they deserve sooner or later? Unfortunately, the truth is that there’s a massive amount of money involved in the battle for reviews, which can have a monumentally negative impact on gaming communities.

Drowning the Market

Recently, there was a big-time scandal about rating manipulation for profit on Kongregate, a significant game publisher. Here’s what’s happened.

Based on user ratings, Kongregate awards cash prizes to the “best” game of the week and month, to the tune of over $7,000 a month. Sounds fair enough. After all, there’s nothing like a good old-fashioned competition to keep things interesting, right?

Here’s the catch. For developers able to churn out attractive but low-effort content, it’s quite lucrative to promote all of your content quickly, while slowing down that of the competition. It’s simply a numbers game, where you increase your odds of success by saturating the community with your content.

Reportedly, over $8,000 was illegitimately earned in just the last few months before the fraud was detected. That’s not really small potatoes. Typically originating from countries with lower standards of living, these seemingly small prizes go a long way, especially when they’re cashing in on a weekly basis. Unfortunately, those who perform these dishonest strategies are extremely skilled with computers and can do anything from buying old accounts to performing algorithmic attacks with “surgical precision”.

So how can the gaming community defend itself? The most effective approach is simply understanding where the attacks are coming from, what affect they have, reversing the damage, and improving existing systems to deter similar behavior in the future. It’s really the best we can do. Or is it?

A Possible Solution

Here’s an idea. Emulate Rotten Tomatoes’ strategy towards reviews: separate user reviews from critic reviews. Sure, there’d need to be workarounds to ensure quality reviews within an accessible community But it wouldn’t be difficult to create an additional “level” of user... something along the lines of Yelp Elite members.

You can still require the bare minimum for basic user verification (to at least slow down most account automation), but make “Elite” user status something to strive for. Quality reviews, a history of (verified) game downloads, and regular community activity could be basic requirements, but acceptance into the program shouldn’t be automatic.

The bottom line is, as soon as you design rules to automate a system, someone will figure out how to abuse them. The best line of defense is to create and maintain a busy community of basic, “elite”, and actual critic reviews.

Because at the end of the day, reviews aren’t going away. Gamers will always want to hear what gets the next.

Topics: Gaming

Related posts

Subscribe to the Community Corner Newsletter and get expert insight and analysis on how to get the most out of your online community every Friday.

Search this blog

Recent Posts

community playbook

Have an Article for Vanilla's Blog?

Send us an email to pr@vanillaforums.com with your topic idea and we'll circle back with our publishing guidelines.