The Future of Data Analysis

Data analysis in gaming companies has gone from being a half-remembered aside just a decade ago, to being the driving force behind some of the world's biggest studios. Every major studio and many smaller ones employ at least one, if not (ideally) a team of data scientists to help it extract information from its games, to analyse and quantify this data and to explain it in a way that can help the studio improve - be that through bringing in more money, keeping people playing longer, getting more downloads or purchases in the first place, or whatever else.

Basically analytics have become as important a factor to the success of a gaming company as just about any other more 'traditional' aspect. But we're still in relatively early days for this discipline - what does the future of the analytical approach hold for game developers, publishers and even players? And, really, what's the point?

Anders Drachen, associate professor at Aalborg University and former chief analyst at Game Analytics, admits this is still an area of the games industry that's growing: "Behavioural profiling in itself is something that has a very well established history in the field of information science, marketing, analytics etc," he says, "So a lot of the methods we are adopting from computer science, information science, into the specific domain of games. But we can't adopt these methods uncritically - games aren't a productivity app, or a website or whatever else: they're games, they're about the user experience.

"As such, there's a lot of adaptation and experimentation happening within this profiling space, just as there's a lot of experimentation happening in prediction. We've been doing this five or six years now for real, across the industry, so we're still trying to figure out how to actually do this thing."

"the richer the information you can get, the more easily you can understand what the players want and build great things for that. It will all feed into better games that people will enjoy more"

Vince Darley, chief data scientist, King

One of the studios that has harnessed data in a big way is King, whose chief scientist Vince Darley - already a true believer in the world of analytics, what with the clear results that studying its player base has brought King - thinks the future of gaming data science is a future of much better games themselves:

"What we're seeing in particular is that mobile games started off extremely simple and have become progressively a bit more complicated, with more stuff to do within them," he explains, "I think in future we'll see more need to understand a more complicated world inside the games, and that will give more insights into the richness of experience in games - the data that we get on it will give more insights into what's actually happening, what the players are enjoying, not enjoying.

"In some sense that will feed on itself as a good cycle - the richer the information you can get, the more easily you can understand what the players want and build great things for that. It will all feed into better games that people will enjoy more. In very simple games you don't really get very much data at all - there isn't much data to gather, it's just did they play or not, did they succeed or fail, that's about it, there's a limit to how much you can understand about a real person from that."

That need to understand what players want is an element in the future of analytics, according to Darley, that won't change any time soon. The need for interpretation of data means that, while there is an ever-increasing, always-improving move to automating data analysis, there's always going to be the human touch involved: "Some industries certainly go overboard with automating stuff, but I don't see the games industry has done much of that," he says, "Your standard tools are non-automated, then there's a number of companies that provide some level of automation - like Game Analytics, Delta DNA, Flurry, Swerve and so on - certainly those companies provide some quicker and more automated ways of interacting with and modifying your game, which can be beneficial I would think.

"But I see that most of the industry is still being very thoughtful and careful and considered about it; most stuff is still at least semi-manual, there's people in the loops trying to understand and interpret, trying to make the right decisions. If anything I'd say there's scope for more automation, so maybe those companies I mentioned will do better in future."

"I think we should stop seeing sales and marketing people as some kind of idiots who don't actually understand anything about what you're doing."

Alessandro Scoccia Pappagallo, analyst at Google

Not everyone thinks it's as simple as 'hire more data scientists and use more tools', though - Alessandro Scoccia Pappagallo, product quality analyst at Google and former data analyst at Square Enix, thinks a big part of the future lies in being able to teach people a wider array of skills. "If you ask everyone what is a skill that they think is most important, everyone's like 'analytics' or something related to data," he says, explaining that a multidisciplinary approach is the best for any business - every employee knowing at least a little bit about different aspects and areas of the business.

At the same time, he knows it's not something that's an easy sell: "The best marketing people are those who know about stats, who know about data. I've met marketing people with an excellent understanding of data," he explains, "I think we should stop seeing sales and marketing people as some kind of idiots who don't actually understand anything about what you're doing. I think that we should try to look at ourselves - how much do we understand about marketing? How much do I understand about sales? I know nothing about sales. So if someone is going to talk about sales with me, I'm not going to be able to understand. Not because I'm stupid or because I'm lacking something in my brain, but simply because it's a thing I don't know about. Once you've shifted to this mindset it becomes simple to understand that you can teach people how to do things. In this regard, one thing I really hope happens more in the future is more communication with HR in terms of development. One of the reasons you have an HR department is not just to hire and fire people - it's to help them progress in their career development. I think it's essential that analytics get incorporated into that process.

"Everyone should be asked when they join a company - even after they've been there a number of years, actually - what sort of skills they want to develop. How they want to develop. Why they want to develop. Otherwise people are just going to be doing the same thing over and over again, which is terrible. I think there's truth in that you should always be challenged in whatever you're doing." One area that might surprise those already in the realm of gaming data analytics is the ability to utilise information for the direct benefit of players - one specific example touted by Drachen being the training of DOTA 2 players using an automated, replay-analysing tool he and associates have developed. He explains: "Basically we developed and algorithm that takes a replay file and automatically identifies the segments of the game where, first of all, something interesting happened - a player was killed etc - and also these encounters are very predictable in terms of outcome of the match. We use this system and with about 80% accuracy predict the outcome of the match after nine minutes."

This, of course, could result in some fantastic tools for training players both casual and professional - and that's something Drachen is very aware of: "Training is the purpose of it," he says, "It's about two years worth of research - it's substantial stuff. The reason why we engage with eSports analytics is that we wanted to find out how we could take this mass of data - both Riot and Valve have systems whereby you can get data, and the community loves stats, we have professional stats people that commentate, all of the pro teams have an analyst - and because it's a digital game we can catch so much data. Whereas with a soccer match, yes we can place cameras around and get some data from a small number of matches - but here we have it for every match. There's so much stuff we can do there, but for the regular player, in order to get feedback on their performance they need a system like this.

"Basically we developed and algorithm that takes a replay file and automatically identifies the segments of the game where, first of all, something interesting happened"

Anders Drachen, Aalborg University

"What we're trying to do now is find the funding - to go an develop this system into something like a web application where you upload your replay and it spits out the information for you, so they kids playing in their team can get high level feedback on their performance. I guess professional teams would also be interested, but they probably have this kind of stuff already, I honestly don't know." So the future of analytics is one that needs a broader skillset, will make games better, can benefit players directly and will still require hands-on interaction from real, actual humans. But does it really need that last part? Sometimes, you just have to ask about artificial intelligence: are the data scientists about to get replaced by the ever-improving AIs of the world? No, in short. "I'd say we're a long way from that," Darley laughs, "It's amazing, the advances that have been made from the deep learning approach, but only on well-defined problems - for extremely hard, well-defined problems like learning to play Go, or doing really amazing image recognition stuff, it's amazing. But that's very well-defined stuff. What we're doing is interpretation - that's not a well-defined problem, so I think people will be the mainstays of that for quite a bit longer."

Drachen agrees, for the simple reason that AIs are made by humans, and right now humans aren't good enough at making AIs: "You take data and put it into them and there is an output," he explains, "But the neural network has to be constructed, there's a lot of variables and features that need to have decisions made about them, it has to be trained and so on. All of these things require human input. Every step of the chain there is the potential for making bad decisions and introducing bias.

"Whether in the future we will have AIs that are so intelligent that they can replace the human input, I don't know. I'm not enough of an expert to tell you. But at least for the foreseeable future, you really need that human element in any kind of analytical work that you do. One key point is that because a central focus in game analytics is the user experience, we as people understand user experience - I don't know if we can generate an AI right now that can. But the guys at Google and Facebook, they can figure out the next step!"