What drives a review score?

During the course of a video game review, reviewers critique a broad array of game design factors: from the main character's clothes to the extensiveness of player agency and choice within a story. But which factors do reviewers spend the most time discussing, and which truly drive the final score?

In order to better understand reviewer behavior and analyze which aspects of video game design drive review score, EEDAR's research team collects and dissects console video game reviews across many different platforms and genres. Once reviews are collected, the research team analyzes the review text and identifies the specific positive and negative callouts the reviewer makes about the reviewed title. These callouts are categorized as one of 63 unique game aspects that fall under 8 primary categories: Gameplay, Market, Social, Narrative, Graphics, Technical, Value, and Audio. These categories focus on the following gameplay aspects:

EEDAR tracks the following data for each review callout: Callout Text Length, Callout Sentiment (Positive or Negative), Callout Aspect, and Callout Category. EEDAR uses this review database to help game developers better understand what factors have strongly influenced past reviews for titles comparable to their own. In this article, EEDAR shares three takeaways from the review data.

1. Reviewers focus most on Gameplay, but they also critique a broad range of other game aspects.

One of the keys to understanding what drives reviewer sentiment is simply understanding what topics reviewers spend the most time discussing. The following pie chart illustrates the average amount of text within a review allocated towards discussing each category.

Gameplay discussion receives the largest allocation of critical review text, as reviewers tend to be very descriptive and explanatory when discussing how a game functions. Within this category, callouts explaining mechanics and the feel of a game's combat systems account for the largest amount of text.

Across the seven other categories, reviewers direct a more even spread of attention. While reviewers focus more strongly on categories they feel are important (such as Market and Social), categories such as Value and Audio still receive an appropriate amount of callout text to discuss their strengths and weaknesses.

2. Narrative Aspects in games remain a pain-point in reviews.

Another interesting application of the review database is to understand general game factors that the reviewer community broadly feel games are "getting right." EEDAR looked at the percentage of callouts that were positive for each category. Based on this approach, the graph below shows areas reviewers feel games are more often succeeding than not.

Across reviews, graphics are discussed in a mostly positive light. Reviewers also have generally positive things to say regarding social elements, audio, and gameplay features. While games continue to provide strong Graphics and Social offerings, titles that do not differentiate from current market leaders are heavily scrutinized (noted within the Market category). Market is the second-most discussed topic with generally negative sentiment.

Narrative and Technical are areas that have grown more negative in recent years. Narrative Aspects in games are mostly criticized and placed against high standards in reviews. Reviewers often feel that the advancement of Narrative quality has been lagging behind that of other key game aspects such as Gameplay, Social, and Graphics. Bugs, connection problems, and other aspects within the Technical category have been growing in frequency for new releases, leading to a growing negative sentiment.

"Even for titles where multiplayer is a key staple of the experience, overall sentiment in reviews is primarily driven by the story-based campaign experience"

Review callouts for Graphics being very positive demonstrates not only development teams' success in creating visually appealing titles, but it also speaks to the diminishing returns graphical improvements have been seeing in recent years (as reviewers tend to be satisfied by most games' graphical achievements). This signifies an opportunity for developers to start shifting more focus onto other key areas of development such as Gameplay, Narrative, and Technical Aspects, which are categories that receive more mixed overall sentiment in reviews.

3. Single-player ultimately drives review score, even for titles with strong focus on multiplayer.

So if Reviewers cover a broad diversity of game factors and generally feel that graphics are often done well while narrative is lacking, what review factors actually impact the score of a specific game the most? EEDAR combines total text allocation with a rating of sentiment severity in order to more accurately understand how that point of discussion is impacting the final review score across a set of games. The strength of impact for each category was indexed using Gameplay as a base.

Market factors (i.e. comparisons to market leaders, comparisons to the franchise's previous iterations, and how well the title lived up to its own expectations) are the strongest drivers in impacting review score. These discussions have twice the amount of impact in review score over general Gameplay discussions. The game's plot is also crucial to review score; Narrative progression holds a strong weight across all titles that include a story mode.

Despite being the third-most discussed subject (13 percent), multiplayer elements (Social) ultimately hold a low impact in review score. Despite significant multiplayer features in titles such as Assassin's Creed Unity, reviewers tend to discuss multiplayer features in a more explanatory fashion without strong opinion. The likely explanation for Social's lack of importance is that most reviewers play a title before it is released and without an active community. This does not allow the reviewer to fully experience the quality of multiplayer features.

Even for titles where multiplayer is a key staple of the experience, overall sentiment in reviews is primarily driven by the story-based campaign experience. For example, the campaign for Call of Duty: Advanced Warfare was found to be significantly more impactful than the multiplayer elements for review score.

Unsurprisingly, Call of Duty: Advanced Warfare's Combat Systems (with the inclusion of the new EXO suits) played the most crucial role for reviewer sentiment. This aspect was found to be 2.24 times more influential in reviews than the 10th ranked aspect, Quick Time Events. The game's story mode was often discussed: aspects such as Campaign, Characters, Dialogue, and Onboarding Process were found to be very influential in reviews. Conversely, the only dedicated Social aspect found to be within the top 10 most influential aspects was Non-Campaign Co-op (the EXO Survival mode). Aspects such as Skirmish, Matchmaking, and Multiplayer Mode Diversity were not within the top 10 most influential aspects.

As multiplayer and general social offerings continue to grow in importance across games, it is important for developers to be aware that reviewers still primarily focus on the single-player experience. Currently, the best way to incentivize reviewers to focus more on multiplayer offerings is by providing the game for review once there is an active community playing the title.

Since a high review score is important for generating unit sales, it is crucial for developers to understand which factors ultimately drive review score. The importance of individual game aspects changes by genre and even by franchise, which makes it even more important for developers to understand how reviewers view their title within the context of the overall market.