Metacritic debunks score weighting report
"Their guesses are wildly, wholly inaccurate" says score aggregator site
Metacritic has been quick to dismiss a report that attempted to calculate the weights that the score aggregator site attributed to different publications, calling the article "wildly, wholly inaccurate."
"Today, the website Gamasutra 'revealed' the weights that we assign to each gaming publication (for the purpose of calculating our Metascores), based on a presentation given at the Game Developers Conference this morning," it said in a statement yesterday.
"There's just one major problem with that: neither that site, nor the person giving the presentation, got those weights from us; rather, they are simply their best guesses based on research (the Gamasutra headline is misleading in this respect)."
The article that the statement refers to was originally titled Metacritic's Weighting System Revealed, and was based on a GDC session by Adams Greenwood-Ericksen of Full Sail University called A Scientific Assessment of the Validity and Value of Metacritic. It listed publications in six tiers, from highest to lowest, according to the importance their scoring had to a game's metascore.
The Highest category included The New York Times and The Official PlayStation Magazine UK, while the Lowest was home to titles like Jolt Online Gaming and Play UK. The piece caused controversy on Twitter as media outlets checked their ratings, and holds a special interest for developers whose bonuses can be linked to a specific Metascore and publishers whose stock can rise and fall on the site's output.
Metacritic argues that actually it has far fewer weighting tiers than suggested, and that the the study has omitted, under valued and over valued a large number of publications.
"The disparity between tiers listed in the article is far more extreme than what we actually use on Metacritic," it added.
"For example, they suggest that the highest-weighted publications have their scores counted six times as much as the lowest-weighted publications in our Metascore formula. That isn't anywhere close to reality; our publication weights are much closer together and have much less of an impact on the score calculation."
Greenwood-Ericksen has yet to comment on Metacritic's comments.