Steam adds histograms to address review bombing
Ability to see how user reviews change over time one step on the way toward prediction-based review scores
Review bombs have become a common occurrence on Steam, and Valve is looking to change the way it handles them. In a post on the Steam Blog, Valve's Alden Kroll laid out the company's thinking on the practice, where players leave negative reviews for a game en masse in order to push the review score down and hurt its financial prospects.
"So why is review bombing a problem?" Kroll asked. "On the one hand, the players doing the bombing are fulfilling the goal of User Reviews - they're voicing their opinion as to why other people shouldn't buy the game. But one thing we've noticed is that the issue players are concerned about can often be outside the game itself. It might be that they're unhappy with something the developer has said online, or about choices the developer has made in the Steam version of their game relative to other platforms, or simply that they don't like the developer's political convictions. Many of these out-of-game issues aren't very relevant when it comes to the value of the game itself, but some of them are real reasons why a player may be unhappy with their purchase."
In the cases where the review bombers were reacting to an out-of-game issue, Kroll said the act usually caused a temporary distortion to a game's review score. But in cases where the scores didn't return to their prior level, Kroll said, "we believe the issue behind the review bomb genuinely did affect the happiness of future purchasers of the game, and ended up being accurately reflected in the regular ongoing reviews submitted by new purchasers." Whether developers changed their game in response to the review bomb didn't have any correlation to how their review score changed after the review rush was over, he added.
"In short, review bombs make it harder for the Review Score to achieve its goal of accurately representing the likelihood that you'd be happy with your purchase if you bought them. We thought it would be good to fix that, if we could do it in a way that didn't stop players from being able to voice their opinions," Kroll said.
The fix Valve settled on was to add a user review histogram on every game's main Steam page. The histogram charts a game's ratio of positive-to-negative reviews since its launch, and lets users click on a time period to see sample reviews from that time.
"As a potential purchaser, it's easy to spot temporary distortions in the reviews, to investigate why that distortion occurred, and decide for yourself whether it's something you care about," Kroll said. "This approach has the advantage of never preventing anyone from submitting a review, but does require slightly more effort on the part of potential purchasers. It also has the benefit of allowing you to see how a game's reviews have evolved over time, which is great for games that are operating as services."
The new information also needs to be put into slightly different context, as Kroll said review scores on most games generally trend downward over time as those who are most interested and likely to enjoy a game tend to purchase it closer to launch.
As usual with Steam, this tweak is not the final word on the subject, but one thing Valve is trying en route to another solution it has in the works.
"It's quite possible that we'll need to revisit this when we move to personalized review scores, where our prediction of your happiness with a purchase is based upon the games you've enjoyed in the past," Kroll said.