An introduction to games user research

Playtesting is hardly a new discipline in games. We actually dedicated an in-depth guide to the job.

But over the last 15 years, the field has matured, leading to the emergence of more specialised areas including user research, that coexists peacefully alongside other forms of playtesting.

"What games user researchers do is they essentially professionalised playtesting, so they take the expertise from academia, or broadly science and user experience, and they use that to design high quality studies where the results are reliable, and we're able to make important gameplay decisions based on the results of the studies that we run," explains Steve Bromley, freelance games user researcher.

Bromley has worked in the field for over a decade -- at PlayStation for five years on titles such as Horizon: Zero Dawn, and with indie developers. He's also created a games user research mentoring scheme, which has partnered over 100 students with companies such as Sony, EA, Valve, Ubisoft and Microsoft, over the last five years.

"User researchers run playtests to make sure players are experiencing games in the way in which the designers expect them to"

"User researchers run playtests to make sure that players are experiencing games in the way in which the designers expect them to," he continues. "When designers are making their decisions they have an understanding about what they think players are going to do and understand, what they think they have to tell players, whether players will notice the right things. But it's not possible to tell if that really works without actually getting it out in front of players."

User researchers are not here to tell developers how the game should be modified based on the audience's reaction; their aim is rather to make sure the vision and the result are aligned. User research is also different to QA -- it's not looking at whether the game is technically broken.

"I think people get worried when they hear about testing things that our objective is either we're going to make the game easy because [players] say it's too hard, or we're going to change the spirit of the game. They might get it confused with market research or focus groups where you get a whole bunch of players in a room and you say 'do kids like skateboarding games this year or do you like games about hedgehogs?' and then come back by saying 'oh, you should make a game about hedgehogs because kids love hedgehogs this year.'

"We're very careful to understand what the designers are trying to do, we're not trying to change the vision"

"Neither of those are what we're doing. Instead, we're very careful to understand what the designers are trying to do, and then make sure that we are only saying: are you doing the thing you're meant to do? We're not trying to change the designers' vision. So if your game is meant to be hard, if it's something like a Bloodborne or Dark Souls, we understand it's meant to be hard, and then we'll check [whether] the game is the right kind of hard, rather than saying: this game is too hard, let's get rid of it. And again, if you want to make a game about skateboarding, that's cool, we're not going to tell you 'don't make a game about skateboarding,' it's not our job to say what the design should be, it's just: has it been done correctly? Have you made what you think you're making?"

What does user research entail in the day-to-day?

On a day-to-day basis, a user researcher is either planning, running, or debriefing a study. These studies can happen at any time during development, though Bromley notes it's usually in the latter half of the process that his services are typically needed.

"What a study involves at the beginning is working with our design colleagues, producers, and other people who are making these decisions, to understand both the current state of the game and what questions they have," Bromley explains. "What are they unsure about? What have they recently been working on that they're not sure if it works or not? And define some objectives, so what do we want to learn from this playtest or this study?"

From that point on, the study needs to be designed carefully against the objectives mapped. There are a wealth of different study formats, from one-to-one sitting with players to watch them play and observe whether they are understanding or not, interviews to see what they're thinking, or more analytics-based approaches.

The latter for instance could use telemetry and measures from the game to see if players are completing the level in the amount of time you expected, if they're failing the amount of time expected, and looking out for whether the experience they are actually having is the experience the designers think they should be having.

"We have to ask [players] questions, we have to set them tasks, we have to be careful to apply those scientific principles so that we're not biasing the players and we're getting realistic behaviour that you would see in a real player at home," Bromley explains.

"Then we've got a lot of raw data, all those observations we made in our session, interview notes, all the telemetry from the game itself, and we'll have to crunch that data into some conclusions, because everyone else is much too busy and working on actually making the game to look at raw data."

Typically, user researchers would then write a report or a presentation, or prepare a workshop that explains the potential problems with the game, and things that players didn't understand, helping designers to make the right decisions, whether that's changing a tutorial that didn't successfully teach players how to use an item, or remapping a level so it's clearer where players need to go.

"Essentially, [it's about] helping a team make that step to want to know what's wrong and to think about what they should do about it," Bromley says.

What education and experience do you need to become a user researcher?

A lot of user researchers come from an academic background, and often have studied a type of social science like psychology, or "something where you have to interact face-to-face with real people, and run a study," Bromley says.

"I didn't know games user research existed and I happened to be doing a postgraduate course in human computer interaction, which is about usability and how people understand how to use software," he continues. "And I was very lucky in that there was a games usability lab in Brighton -- where I was studying -- and one of the lecturers owned that lab, so I got first time experience through that.

"But beyond my own experience working that lab, it is very common for [user researchers] to have done an undergraduate or a postgraduate degree in some sort of science, and have been interested in: how do I design a study? And can I interview a person or do a test on a person, and then look at how you can apply those [learnings]?"

"Being a user researcher in any field is a good way to transition because a lot of those skills about how you design a study are applicable"

Combining this social science background with an interest for games, how they're made, and what constitutes good game design, is usually how people join the industry as user researchers, Bromley adds.

Education aside, knowing where to find work experience in that field is a common question Bromley faces when running his mentoring scheme.

"There's a couple of things that we recommend. One thing is: being a user researcher in any field is a good way to transition because a lot of those skills about how you design a study are applicable. But for games, one of the activities we set people to think about getting experience for real is working with the indie community.

"There are a lot of places like the Reddit playtesting forum or Itch.io playtesting forum where there are people who have made games and they are looking for playtesters for feedback. They don't necessarily know that user research exists, or that you can do this in a scientific and a measured way, but the fact that they are asking for help and they have a game that is in development means they're in the right state where they can benefit from your help.

"And so we often recommend that people who are interested in this field reach out to indie developers who are interested in getting feedback on their game and explain: hey, I'm not just going to be a play test participant, but actually I can apply some of this experience I've got either from academia or I've done some reading around it, and I can give some advice based on usability best practices, and use that as an option to run a study. I think that's invaluable experience."

"Understand how games actually get made, and then also do some learning about how to run a usability study"

Bromley also recommends doing some reading about game design, such as The Art of Game Design by Jesse Schell.

"Understand how games actually get made, and then also do some learning about how to run a usability study," he continues. "There are a couple of good books for that -- Just Enough Research [by Erika Hall], and Don't Make Me Think [by Steve Krug]."

Bromley himself also wrote a book on this topic, called How to be a Games User Researcher.

"Doing that reading to understand how [to] run a study, doing some reading about game design, actually reaching out to indie developers -- that makes you a fantastic candidate so when these jobs come up you've got all the practical experience and theoretical knowledge to be greater doing it."

What skills do you need to be a good user researcher?

Being a user researcher means spending a lot of time talking and interacting with people. So at the top of the desired skills is this social aspect and an ability to communicate effectively.

"Can you talk to a player, can you have a conversation with them, but also doing that in an unbiased way? So you're not giving your own opinions, or you're not leading them to any answers, but creating a space where they feel comfortable giving their opinions, and explaining what they think, and if they get stuck they don't feel like they're being stupid.

"That social side also applies to the teams you work with, so you have to spend a lot of time talking to games designers to understand what their game's intention is and, also, at the end of it, you've got to convince them that maybe it's not working, so you've got to have those conversational skills to work around that."

"Can you talk to a player in an unbiased way? So you're not giving your own opinions, or you're not leading them to any answers?"

An analytical mind is also very important. You need to be able to convert an idea such as "I'm not sure if my tutorial works," presented by the developer, into tasks you can set for a player, questions to ask them, and figure out a way to know whether they understood everything properly.

"Being able to apply that analysis to break a vague objective down into concrete tasks is a really core thing that you have to do," Bromley continues. "And then doing that same analysis on the other side. You've run your study, you've got a lot of data, and again, no one's going to pay attention to all that data. Using your analysis skills, you have to turn that raw data into 'Here's five really interesting and really well explained points that someone's going to listen to and do something about'."

What's the state of recruitment and opportunities for career progression?

User research roles come in a mix of in-house and agency opportunities. Bromley points out that it's "reasonably difficult" to find a job at a junior level.

"I think once you've hit mid-level or senior, there's loads of positions, because it's a growing discipline that isn't yet considered a core part of how people make games. So only the largest studios have teams, and smaller studios -- if they have anyone -- they're only going to have one or two very senior researchers, they're not going to be looking for people at junior level," he says.

"That obviously makes it very challenging, and that's why we recommend, as well as getting that academic and theoretical experience, actually doing some work with indie developers. And a lot of people won't do that or won't take it that far, so [if you do] it means that you are an exceptional candidate when you turn up to the interview; you are a lot more likely to get that junior role because you have done so much to prepare for it."

"Some people use this to transition into an accessibility focused role, if you're interested in making sure that people with different access needs can play games"

In terms of career progression, you can of course make your way from junior, to mid-level, to senior user researcher, and stay in this branch. The difference between the different levels of seniority is that you tend to "take more ownership of more of the project," Bromley says.

"At your early stage, you might just be running a few tests at the end of the game, and as you grow in experience and your relationship with your teams grow, you get to have more influence over earlier in development, and run those studies earlier as well, which I think is really important," he adds.

User research can also give transferable skills that make it easier to branch out in other roles.

"Some people use this to transition into an accessibility focused role, if you're interested in making sure that people with different access needs or different disabilities can play games.

"And I've seen people going into more general production roles, because being a user researcher means you have to understand what everyone else is working on, so that you can run your studies. That can help make you a very good producer because, again, that needs an understanding of all the disciplines and bringing all of that together."

"Making sure that you're aware of the community, and you're actively reaching out and taking advantage of all the help that's out there, is really important"

If you want to learn more about production, the GamesIndustry.biz Academy recently published a guide to the field right here.

Concluding our chat, Bromley points out that the user research community is a small one, but it's very active. So if it's a field that interests you, reach out to peers.

"There's a group called The Games Research and User Experience Special Interest Group and they have a very active Discord, they put on events, conferences and networking things," he says. "It's really nice because there's not that many user researchers out there and so you do get direct exposure to people working at all the big companies, the current discussions are happening in the industry, and a lot of support if you are joining the industry to do things like reviewing CVs or reviewing portfolios. For example, the mentoring scheme is part of this community.

"Making sure that you're aware of the community, and you're actively reaching out and taking advantage of all the help that's out there, is really important."

More GamesIndustry.biz Academy guides to Working in Games

Our guides to working in games cover various perspectives, from hiring to retention, to landing the job of your dream or creating the right company culture:

• How to get a job in the games industry
• How to not ruin your job interview
• How to make your video games resume, portfolio, and job application stand out
• More articles about working in games...