Deep UX: The last "Here Be Dragons" area of modern game development

In this article I will touch upon the integration of "deep user experience" in a modern mobile game project that allowed me to achieve a significant and constant growth in retention. By "deep UX" I mean a complex of measures aimed not only at tinkering with such known values as flow, user expectation management and interface convenience, but with more complex things such as emotional involvement and retention.

This story started as a job interview with a potential developer. As it often the case with such meetings, we'd been talking about my studio's product lineup. We started to discuss our shoot 'em up HAWK as an example of a recent successful launch. The job candidate didn't share my high opinion of HAWK: he said that he'd tried the game but found it boring, and suggested a rival shoot 'em up as a good example of the genre (although this game was less successful on the market).

As a game designer, I have a hand in all of our projects and listen carefully to what end users have to say about my products. This feedback stuck in my head. For about a month I mulled over this criticism, considering its fairness. On one hand, I had a product with excellent market values and genre-topping revenue. On the other hand, this person said our game was boring and preferred a rival title with much less revenue.

I tried to decide if it was a problem that I'd be able to solve or just a fluke - a bad review not worth paying attention to. The game was successful, with high retention levels and lifetime value ratio; did I really need to try and fix what wasn't broken in the first place?

The only thing that would put my mind to rest was a comparative UX test, as we have always taken UX seriously and had both the resources and the tools that this sort of study required. The crux of this study was the search for a perfect 'mood' that users are looking for in the beginning of a shoot 'em up game. For this, we decided to compare the user experience HAWK vs. two rival titles with lots of excellent reviews: Sky Force Reloaded and Fastlane: Road to Revenge.

All of our respondents played each game for 15 minutes in a random order. After each title, the respondents completed the Game Experience Questionnaire and Microsoft Desirability Toolkit and discussed their experiences with a moderator. The results of the study proved to be very interesting and confirmed the job interviewee's feedback: all the players considered our game the most boring of the three.

We also had a weird negative-to-positive affect ratio.

Even though HAWK's positive affect is lower than the other two games, the negative affect is also minimal. It suggests a cautiously positive impression of our game: the players weren't blown away by it, but they also didn't respond as negatively as to the rival titles.

The most interesting result was the Flow, Tension and Challenge ratio. HAWK scored very high in Flow (the ability to immerse the player and hold his attention), but both Tension and Challenge scored so-so.

I had managed to acquire hard statistical data to back up the random feedback from the job interview. Despite my product's commercial success, the players' attitude toward it was much cooler than toward the rival games. As a producer, I saw a possible growth point. I thought about how, by compensating the initial coolness during the immersion phase while retaining the simplicity that was our strong point in comparison to the competition, we could grow our retention level even higher.

But what is this "cautiously positive" response to our game? How can we turn it into an excitedly positive reaction while maintaining the minimal level of negative affect? I didn't know the answer, so I turned to my team for a discussion.

"The game was successful, with high retention levels and LTV; did I really need to try and fix what wasn't broken in the first place?"

I still remember the meeting. During my many years in game development I have attended so many product meetings that I thought nothing could surprise me anymore, but this meeting did. I simply found myself unable to explain the problem clearly and chop it up into specific tasks for my team members.

As I wasn't able to quickly articulate the problem and explain its importance, the developers turned hostile and formal. I think I wouldn't have been able to turn them around if not for that playtest with its clear numeric values. They couldn't ignore our lesser numbers in comparison to our rivals - even when considering abstracts like tension or positive affect.

After the rejection phase we doubled back to constructive thinking, but we still weren't able to separate the problem into specific tasks for each member of the team to solve - until we stumbled upon a very simple idea that immediately turned into a solid plan of action.

The basis was the fact that, in HAWK, we learned how to retain and monetise the player for an amount of time that was unique to the genre. Our LTV is based around six-month periods, which means that even after half a year in the players are still interested enough to keep paying us. We decided to focus on the experience that allowed us to retain our players for such a long period and hyperbolize it at the game's starting, immersion phase.

I'm only going to touch lightly upon the log of our changes because, in fact, we changed pretty much everything. We cut the dedicated tutorial and made the first combat progression level our learning phase. We turned narrative into gameplay to minimise the player's contact with the non-combat interface during the first session.

"I'm only going to touch lightly upon the log of our changes because, in fact, we changed pretty much everything"

We changed the ability unlocking system to give the player a heavy use of abilities right from the get-go. With our simplest, starting fighter we tried to keep its simplicity and ease of use, but added more design charm and dynamic firepower. And, of course, we completely rebuilt all of the beginning levels using everything we had learned since the launch of the game.

As the problem we tried to solve was an abstract with no concrete proof of its huge significance, we did all of this parallel to our main development process. It took us three months.

In the end, after implementing all the changes, we compiled an 'un-boring' version of the game and tried the UX study one more time. Imagine our reaction when we found out that we became leaders, not outsiders, in terms of emotional involvement. Now, nobody called our game boring; on the contrary, the respondents noted its charm and quickness while still praising the simplicity that was missing in our competitors' games.

It was a victory. But was this increase in general quality of the game enough to raise our already good retention levels?

In the beginning I was skeptical. During my 13 years in game development I'd seen many a project and patch aimed at increasing some market values, including retention. But there hadn't been a single case of actually being able to raise a good retention level. In every positive example a game with low retention was able to increase retention through technical optimisation and by fixing major conversion funnel fails. In our case the project was well polished, and beyond such critical mistakes.

So when we implemented the changes and saw our retention increase, we felt like we'd completed a super-hard quest in one of our games. It all started as a job interview where we received random and highly abstract feedback on HAWK, but we didn't ignore it. Instead, we chose to commence a study and formalise our general expertise into concrete changes in our game's most problematic aspect.

When the game stopped being 'boring', we received +5 per cent to the classical Ret1. The effect held through the days to come. This is an average effect for all platforms and territories.

Conclusions

For me this example seems extremely telling, as it was the first time I was able to increase retention while working not with classic stats analysis, but with user 'mood' - with deep UX. It's a completely different level of development that is much more complex and demands a special kind of approach. But, its difficulty notwithstanding, I consider this the approach of the future.

"To succeed, you've got to be head and shoulders above your competition, and for this you need data that your rivals don't possess"

I launched my first mobile title, Juggernaut: Revenge of Sovering, in 2012. So far in 2018 I've launched six games with three more in prep stages. Looking back at 2012, I see the way the industry has advanced in the last six years. I see how technological and intellectual components of mobile game development have grown, the development itself becoming more of a complex, branching structure, how many new specialisations have appeared, and how the density of knowledge has increased for every megabyte of content and each new user.

Paradoxically, though, it's become much simpler to make games. All the required tech is available to a casual user, and tech specialists are also widely represented on the market. The majority of genres have already been formed, and their gameplay mechanics are researched and studied. New mechanics are quickly prototyped and tested on live users. Simultaneously, a ton of new analytic tools and well-established practices have appeared. Everyone knows what to look for, and how, and what's normal or what's procedure. Game development knowledge is now open source, and it has never been this quick and transparent.

But this model has its dark side. As this knowledge is available to virtually everybody, it's never been this hard to lift yourself above the crowd. In comparison to 2012, there are many more games at basically the same level of quality both technically and in terms of gameplay. To succeed, you've got to be head and shoulders above your competition, and for this you need data that your rivals don't possess.

I think this is UX data. This knowledge belongs to a highly abstract plane of emotions, expectations and other unquantifiable constructs. After acquiring this data, it can't be copied as it is irrevocably linked to a particular product. The knowledge in itself is useless; the value lies in the methodology of acquiring and processing these data sets. Developers who are able to do this will have a significant market advantage over their rivals.

Deep UX is one of the last 'Here Be Dragons' areas of modern game development. Any person with concrete data on this will be able to achieve market advantage in the next few years.

Aleksandr Enin is the head of IT Territory game studio at Mail.Ru Group.