Design Research: Ignore it at Your Peril

Everything Peter Molyneux says is wrong. Maybe not everything ever, but certainly all the key points that were reported from his keynote speech at the recent G3 Futures Expo - they are simply not true. To be clear, I'm not saying I have a different opinion to Peter, but rather, the best knowledge that exists today on design and creating successful products is completely contradictory to the advice that was given. Let's take a look at two key statements that were made and see why this is important for designers, and in fact everyone in the games industry, to consider.

Statement 1 - "My games would be a quantum times better if I had experienced eyes looking at them." Here Peter states that the best people to test new titles may be fellow developers, and he hopes that those already in the industry will be better at 'spotting problems' with in-development titles. There are two problems with this idea, firstly the value of experienced eyes, and secondly, who those eyes belong to.

The first problem, that of being experienced I've discussed before, the amount of years on the job is not an accurate indicator of identifying people who are good at their job, it's not years of practice that matters, but whether or not they have engaged in what is called deliberate practice. What this comes down to is that it's not just about how much you practice, but how you practice. What this means is that choosing 'experienced' eyes will be very difficult indeed, simply going for people who've been around a while may be of little value, whereas a relatively new developer who has followed the principles of deliberate practice may be much more valuable. Can you easily identify who is who?

"If Peter wants to get ideas from fellow designers then that is completely fine, however if he wants to validate design ideas with designers then that is not fine"

The second problem with this first statement, is that regardless of the quality of their experience, they are precisely the wrong people to get feedback from. If Peter wants to get ideas from fellow designers then that is completely fine, however if he wants to validate design ideas with designers then that is not fine. I've talked before about knowing the intended audience for your game, and one of the reasons why this is so important is so you playtest mainly with those player profiles (others are possible), it's those players you based your design decisions on.

Statement 2 - "We're not competitors, really. There's seven billion people in the world - surely we don't have to worry about competition."

Data released by Google show that the majority of users when searching for an app compare at least one other competing title, so you most certainly should be worried about competition. And yes, there may indeed be seven billion people in the world, but your game is designed for only some of them, and so is your competitors'.

There are two things I can conclude from all this, One, Peter mustn't read my column, and two, he doesn't seem particularly well informed about design. Design is not just about the idea creation stage, or the mechanics of how a game functions, but also how you manage and inform the design process. This is known as Design Research, and is taught as a basic concept on many design-related undergraduate programmes.

"Peter is not alone in making such reckless statements. I was recently at a conference where a speaker was saying that all that was needed to make a great game was to hire smart people and have good ideas"

However, Peter is not alone in making such reckless statements. I was recently at a conference where a speaker was saying that all that was needed to make a great game was to hire smart people and have good ideas. This produced plenty of nods in agreement and I was thinking that everyone who is nodding along right now is probably making fundamental mistakes in their game. I'm not suggesting to not hire smart people or have good ideas, but without a way to manage the creative process, results are likely to be unpredictable, i.e. you may get a good game, but then again, you may not.

These sorts of misunderstandings around established practices in the design process seem to be widespread and I have to wonder - is it because many in the games industry don't know about Design Research, or because they choose to ignore it?

What is Design Research?

Design Research is a collection of methods which can be used throughout the development process to either inform design decisions, or validate design decisions. No matter which stage you are at in development, there's a design research method that can be applied. Broadly speaking these methods can be split into two groups, investigative methods, and assessment methods.

Investigative methods are those which are used to understand your target audience and would typically come at the start of a project. The aim here is to understand why your target players play the games they do, and this player insight helps inform some of your game's early decisions. It's important to note that this is not about asking players what they want, it's about understanding their behaviour.

The second group, assessment methods, typically happen at a stage in the development lifecycle once a playable build is ready. These methods, such as playtesting, are used to validate the assumptions that were made during the design phase. One type of assessment method that can happen early on however is a competitor analysis, which is used to inform design decisions and best-practices from already existing games.

"Perhaps more worrying than not knowing about design research is knowing about it and choosing to ignore it"

I Know Better

Perhaps more worrying than not knowing about design research is knowing about it and choosing to ignore it. This is more worrying because it means you're familiar with a process that is known to produce better games, yet have instead decided to not use it. I've heard many excuses over the years as to why a design research process isn't being used, here's a small sample.

"It's too early in the process to do anything". No, it's never too early, Understanding your users is meant to happen before you begin designing as understanding informs design, this is typically the first phase in most design processes. If you've got to the design stage and haven't done any user research then you're already introduced mistakes by not being correctly informed, not used best practices, or designed for the wrong players.

"It ruins the creative process". Sadly not. Receiving objective feedback and working within constraints have been shown time and time again to produce better results across a variety of creative disciplines. Design research is intended to enhance creativity by encouraging risk to take place, the process is there to catch any mistakes and put them back on track.

"We know what's wrong and we're going to fix it". Not quite. You have identified known issues that you're going to fix, but it's very likely that are also plenty of unknown issues which you are not yet aware of. If the issue isn't known, it'll not be fixed, and it's better to know sooner in development than later.

"Part of me wonders if some developers choose to ignore design research because of the traditional software development process"

"You can't evaluate fun". Indeed, this is difficult, but you can assess if players understand how your game works and if they can achieve the tasks that are required of them by the designer. Both of these issues can have a significant impact on a game's enjoyment, so if players report that your game isn't fun, it's possible to use this indirect measurement approach to get at the root cause of the issue. By identifying and addressing the underlying cause, a subsequent assessment of the amended game should reveal more positive responses from players.

Part of me wonders if some developers choose to ignore design research because of the traditional software development process. The processes used to create software are typically focussed on delivering features, whereas a design research process is focussed on delivering great user experiences. Of course, both of these approaches can live together quite happily, and it's worth reflecting on your own game development process and how much effort is put into each. Are you focussing on features, or player experience?

I agree with Peter on the core concept though, getting people to play your game during development will indeed dramatically improve the player experience, however they need to be the right people. He doesn't need more designers, he needs a smarter approach to making games.