Kunai becomes the latest title review bombed on Metacritic by a single person

Kunai, a new Metroidvania title from developer TurtleBlaze and published by The Arcade Crew, has been relatively well-received since its February 6 launch. It's currently sitting at "Mostly Positive" on Steam, with an 81 Metascore for the PC version on Metacritic.

But on Sunday, February 17, the developer and publisher noticed something odd going on. The game's PC Metacritic user score, which had previously been at an 8.1, had plummeted to a 1.7.

Perplexed, TurtleBlaze investigated, and found that it was the victim of a practice that's becoming increasingly common nowadays on sites like Steam and Metacritic: review bombing. But while review bombing has been commonly used by large swaths of players as a form of protest against matters ranging from in-game issues to a developer's politics, this particular review bomb was unique in that it appeared to have been carried out by a single person.

In a blog posted to Gamasutra, TurtleBlaze programmer Benjamin de Jager explained that the review bomb on Kunai was not the first such incident of a single person dropping a game's Metacritic user score seemingly overnight. In fact, the culprit had originally done something similar to Insurgency: Sandstorm, bringing the game from (per the person claiming to have done it) an 8.1 to a 3.6. Currently, Insurgency: Sandstorm has a 4.1 user score.

"Review bombings are something we see from time to time, but usually not on indie games, and usually they have a message behind it"

Arnaud De Sousa

The culprit took to Reddit in a since-deleted post to explain that they tanked the games' scores simply by making a lot of throwaway email accounts, using those email accounts to make around 200 Metacritic accounts, and then giving the games a 0 user score on each. An actual written review was unnecessary, and there were no other barriers beyond making the accounts and verifying the email address. They simply logged in and hit the 0 score button on the game, and it was done.

GamesIndustry.biz confirmed the ease of account creation and reviewing on Metacritic by making several accounts with throwaway emails. Though there is one visible layer of anti-bot protection on account creation, there was nothing stopping a human from creating several accounts at once from the same IP address using email generated quickly through services like Gmail, Outlook, and others.

Furthermore, according to the culprit's Reddit post, both Insurgency: Sandstorm and Kunai were targeted entirely at random, with no protest or agenda driving the sudden drop in their user scores.

The incident left TurtleBlaze producer Bram Stege and Dotemu/Arcade Crew head of marketing Arnaud De Sousa at a bit of a loss for how to handle the situation, as they told GamesIndustry.biz in an interview this morning.

"We struggled with the tone of how we wanted to address this, because we did not want to attack Metacritic, and we did not want to attack the bomber," Stege said. "Because very quickly, it was apparent that he did it for the attention."

"You've been working for two years on this game. And then one guy with basically no effort can bomb your score from a big, positive score to a 1.7"

Bram Stege

The two ended up reaching out to Metacritic for assistance, but the response took more than a day, and wasn't conclusive. According to Stege, Metacritic promised to look into the situation, and did remove a good chunk of the negative reviews. But some remained, including the original review from the review bomber, which they appeared to have used their extra accounts to upvote to ensure it was prominent on the game's review page.

Now, Kunai is back up to a 7.4, with 19 of its 80 reviews flagged as "negative" (only one of which has any accompanying commentary). However, Insurgency: Sandstorm is still sitting at a 4.1 user score due to 170 negative reviews, only eight of which have any explanation for their bad scores.

Stege speculates that Kunai was particularly susceptible to a review bomb for two reasons: it was a newer game, and a smaller one. As a result, it didn't have that many reviews, so it took very little to drop its score quickly.

"Review bombings are something we see from time to time," De Sousa said. "But usually not on indie games, and usually they have a message behind it. Because developers change something, because the online is really bad -- usually there's an agenda behind it. But here it's just a random dude picking a random game to review bomb because they can."

Ultimately, it's difficult to measure what effect, if any, a review bomb like this has on a game. Though larger companies have in the past allocated bonuses to developers based on a game's Metascore (which is set by critical reviews, not user reviews), the user score is both less prominent and, unlike Steam reviews, not directly visible in the place where consumers might be purchasing the game.

However, both Stege and De Sousa said that the effects of review bombs, especially seemingly arbitrary ones like this, are felt within studios.

"It causes stress," Stege said. "You hope that the audience of Metacritic is smart enough to see, 'Oh, they got an 8.3 from the critics and a 1.6 from players, so there must be something up.' You hope it doesn't affect sales. I don't know if it affected sales. But it does generate a lot of stress. You've been working for two years on this game. You put all the blood, sweat, and tears into it. And then one guy on a Sunday afternoon with basically no effort can bomb your score from a big, positive score to a 1.7. It's very stressful."

Arnaud adds: "And it took us a lot of time to contact people to figure out the right way to address the issue. That time of worry, we weren't updating the game or working on the game itself. It's really time-consuming, these types of issues."

The two also affirmed that review bombs are additionally unhelpful, since (at least at their respective companies) they pay attention to user reviews as a valuable form of feedback for future updates. Review bombs dilute that feedback, making it more difficult for developers to understand whether they really should be making changes, or if the situation is a prank or political protest.

"We're always looking at reviews," said De Sousa. "It's interesting just to have the feedback... When you're working on the game for two years, you get tunnel vision and you don't always see other points of view. Sometimes, we see reviews and it's just someone who played the game 20 minutes and gave it a 0, so it does depend on the quality of the reviews. But someone who played the game for 30 hours and gave good feedback, whether it's positive or negative, it's really interesting to read."

"It's weird to grade a piece of artwork with a score. We're not making homework. We're making a video game"

Arnaud De Sousa

Stege added: "At trade shows, we saw everyone liked the art style and the swinging around in Kunai, and generally people liked the game. But that doesn't mean the rest of the world will like the game. It was an exciting moment to see the first scores given to us and read the first reviews. It's really too bad that the only way to give value to the artistic side of the game is a number. I'd rather have it be something else, but right now we don't have anything else but the number. So we were hoping for a high score."

GamesIndustry.biz reached out to Metacritic to ask what, if anything, the platform intends to do about future review bomb situations, and received the following statement:

"Metacritic takes issues of potential score manipulation seriously and has a number of policies in place to maintain score integrity. Moderators regularly review the site and remove any entries that do not fall within our guidelines in addition to a moderation queue where Metacritic users can flag unusual behavior. Moderators then review and remove any entries that violate our terms of service."

De Sousa and Stege are holding out hope for more, though, both on Metacritic and across other platforms where review bombs are a concern. They agreed that Steam's system of identifying "off-topic review bombs", notifying users when they might be occurring, and showing how much time a user had put into a game when they reviewed it was a step in the right direction. At the very least, Stege said, he hopes Metacritic will reconsider letting newly-made accounts review games instantly after creation.

But De Sousa added that issues like this speak to a broader issue -- which is that numerical scores don't tell the full story of a game's quality, and aren't always the best way to determine whether or not a game is "good" or not.

"It's weird to grade a piece of artwork with a score," he said. "We're not making homework. We're making a video game. It's different."