Valve relents on Steam key reviews - well, almost

Valve has reigned in one of the recent changes to Steam's review system, making it simpler for developers to find individual reviews from users who acquired a game outside of Steam.

In its latest blog post in this "ongoing" process, Valve noted that the difficulty in finding "helpful, articulate reviews written by customers that obtained the game outside of Steam" was a common complaint following the recent changes.

"We want to make sure that helpful reviews can be surfaced regardless of purchase source, so we're making a change to the defaults," the company said. "Starting today, the review section on each product page will show reviews written by all users, regardless of purchase type. By default you'll now see reviews written by all players of the game, including Steam customers, Kickstarter backers, bundle customers, streamers, and other users that acquired the game outside of Steam.

"We want to make sure that helpful reviews can be surfaced regardless of purchase source, so we're making a change to the defaults"

Valve

"Regardless of the default, you may prefer to see only reviews by Steam customers. So we've also made it so that Steam will remember the last 'purchase type' you selected to view in the review section. As you move between game pages, Steam will remember your preference and display only those reviews."

However, Valve isn't relenting on the most divisive aspect of the altered review system. While reviews from non-Steam users will be easier to find, they still won't have any impact on a game's overall review score. "Each game's score will continue to be calculated based only on customers that purchased the game via Steam," Valve said.

Valve's stated aim when it implemented the changes was to combat a common tactic employed to manipulate Steam's review score - which Valve described as "a point of fixation" for many developers. "The majority of review score manipulation we're seeing by developers is through the process of giving out Steam keys to their game, which are then used to generate positive reviews. Some developers organize their own system using Steam keys on alternate accounts. Some organizations even offer paid services to write positive reviews."

Speaking to GameIndustry.biz recently, Slitherine's Marco Minoli acknowledged that manipulation of reviews was a problem being addressed by many online platforms, including Trip Advisor, Airbnb and Booking.com. "The reviews 'issue' is hitting the news because Steam is trying to do something about it," he said.

Indeed, Minoli referred to "a lot of occasions" where competing developers and publishers post, "negative reviews to beat someone else's score down." Slitherine was once a victim of that tactic, a "very painful" experience that resulted in Steam, "deleting all these negative reviews and [taking] action against these people directly."

However, while there may be a problem for Valve to address, the main concern relates to those who might be harmed in the crossfire. Now more than ever, there are entirely legitimate reasons for distributing Steam keys: crowdfunding and the Humble Bundle, for example, or any attempt to create a sustainable community beyond Valve's platform. The most visible legitimate complaints following the initial changes came from those who built businesses through these methods

Slitherine also frequently gives away Steam keys. According to Minoli, the company launches some of its games through its own sites first, then distributes keys when each product reaches Steam. "In fact, we always give a Steam key to redeem for any purchase made on our site," he said. "We just want players to be able to have our games in their library."

"This of course has an impact for us," he continued, "when we rely on our core fans to support the Steam release. These reviews are now lost. That's very similar to what happens to the Kickstarter projects and to other ones that rely on big communities to support games that are niche and need word-of-mouth to be successful."

"The games with a higher number of reviews see basically no variation, both in terms of review numbers and overall valuation"

Slitherine

Slitherine provided GI.biz with data on the impact the initial changes had on every one of its games available through Steam. For games that were first launched on the company's own websites, the fluctuation in the number of valid reviews is striking. Wars of Napoleon lost more than 43% of its reviews, Decisive Campaigns: Barbarossa lost 48.4%, and Last Days of Old Earth lost a massive 64.3%. Out of 45 Slitherine games on Steam, 18 saw a reduction in the number of reviews contributing to that overall score, and all but 4 of those lost 15% or more.

Perhaps the biggest concern is that Steam reviews scores will decline as a result of Valve's changes, and on that front Slitherine's data is less conclusive. While the single largest swing among its games is negative - a loss of 19% for Wars of Napoleon - all of the changes are 7% or lower, with the majority below 4%.

"The games with a higher number of reviews see basically no variation, both in terms of review numbers and overall valuation," said Minoli, who also observed that, "there is an almost equal number of increase and decrease in percentage of positive reviews.

"This situation...is not always negative. For our niche games, for instance, some of the customers buying at lower price [through bundles, for example], sometimes fail to appreciate the depth and complexity of a wargame, [and] post negative reviews. With this new system, we are seeing some improvement in average reviews in many occasions."

Steam Spy has also assembled a list of the 427 games that felt the biggest impact from Valve's initial changes.