Data analytics in a small team: is it worth your time and investment?

Analytics in gaming are a big element of large studios these days, and the relatively young discipline has made its way down the scale to teams of just a few people in recent years. The need to employ some kind of data science and informational analysis on your games is bigger than ever - but it's still tough to figure out if you do really need to bring in a professional data scientist to the team.

Sara Bayley knows better than anyone else how difficult it can be to start up a data analytics wing in a small studio. As data analyst and product strategy manager at Genera Games, she joined the company back in 2015 when it had absolutely zero information-gathering going on: "It was tough," she admits, "Essentially I came in because two people already at the company decided they wanted to start analytics, and they needed someone with analytics experience. So they decided to bring me on. We were working with Game Analytics, mostly because it's free and a way to get access to data, and I just jumped in with one of the games we had in soft launch at that moment, pulling out data and seeing what I could find out from it, learning, moving forward constantly and elaborating on, building off of what I'd started with at the beginning.

"Then there was throwing together presentations, sitting down with the developers and showing them - I think the first time I showed something to the developers they were just looking up at the screen with no idea what I was talking about... So learning where to take a step back and where to dive farther in was important to learn!" There are other factors to consider when thinking about analytics in a small studio situation - not least of which the fact that you're dealing with a lot less actual data, at least if your game isn't bringing in tens of millions of players. i.e. If your game is like the majority of games out there.

"Trying to feel confident in your own results in order to communicate them, then being able to truly show how things are changing is very difficult when you're dealing with less users"

Sara Bayley, Genera Games

Bayley explains: "It's difficult, because a lot of times you're dealing with less users. Trying to feel confident in your own results in order to communicate them, then being able to truly show how things are changing - at least on a monetary side - is very difficult when you're dealing with less users. To show that by making this change users are staying longer - that's something that's relatively easy to show. But on a grander scheme of 'by doing this, I'm making you more money', that's a little bit more tough to show."

But just because you're a smaller studio, a smaller team and dealing with numbers off a smaller variety than the big boys doesn't mean the technique varies wildly - getting analytical foundations in from an early stage can be hugely beneficial for future successes. Vince Darley, chief scientist at King, says the approach to data analysis between small and large studios doesn't change much - it's just the capacity which really alters: "We've got the luxury of having a pretty large, experienced team now, so we've learned more than we would have done if we were a small organisation," he says.

"We're probably at a more sophisticated and mature state of understanding how we can get great value out of the data than a smaller organisation, or more quickly than they would. But otherwise I would say the process is not dramatically different. If I look back at my previous roles in my career where I was in smaller organisations with smaller teams, we'd try and do the same kind of stuff - just obviously you'd get less stuff done and it would be less quick than when you have a massive organisation like King." Alessandro Scoccia Pappagallo, product quality analyst at Google and former data analyst at Square Enix, does highlight a fairly big difference between the larger and smaller companies dealing with data - the fact that the former has a history of acting on things, and so has experience with the fact results will arrive later on, rather than immediately.

"Big companies tend to have a big and strong data science team, they also have a history of acting on data, meaning it's very easy to push through a change"

Alessandro Scoccia Pappagallo, Google

"Something we can all agree is that big companies have very different needs when compared to small companies and startups, especially when it comes to data science," he says, "Big companies tend to have a big and strong data science team, they also have a history of acting on data, meaning it's very easy to push through a change. When you have data supporting your idea, you just have to write out the details, have your code ready and you can just make a change.

"When you work for a small company, or a company that's just not used to that approach, it's way more complicated - and that is where communication becomes essential. You're not going to be communicating just with people that have your same skills and background - you're going to be communicating with people who might have no idea what you're doing."

Pappagallo's solution is simple: educate. Teach people more skills than they currently have, and understanding - both of the core concepts and value of analytics - will follow. It's an approach that is helped when backed up with use of the many tools available on the market, and if possible, by creating in-house solutions to analytical needs.

With the likes of Game Analytics, Delta DNA, Flurry, Swerve and many other ready-made tools available, it's tempting to just stick with the pre-made stuff. In some cases, that's absolutely fine - if not right - and will serve a smaller studio well when there isn't quite the capacity to dedicate a huge amount of time to gathering information. But at other times, building tools internally is something studios should definitely look to do - and it can be done cheaply, if not free.

Pappagallo explains: "In data science people are quite lucky - if you think about other areas, they don't have the luxury of having quite so many amazing free tools. There are a lot of companies now trying to sell their products, and maybe they are useful, I don't know, but you can do so many amazing things just with tools that are free. One of the reasons is that data science, more than pretty much any other area, is very close to academia. In academia money is very limited, so you want to use free tools as much as you can.

"Python is an amazing programming language, I couldn't be more passionate about it, and it's free - you have thousands of libraries, all free. RStudio is an amazing tool for doing statistics, and it's free - at least for individuals, I'm not sure it's free for companies. D3 is a great visualisation tool, and it's 100 per cent free and it comes with a large community of people who can help you with whatever doubt you may have. Those are normally the tools that I would suggest to anyone."

The choice to employ someone to create tools over using those available publicly is another concern, according to Bayley: "If you make a decision that you want to build something internally then you need a very different type of person than if you're going to use something completely third-party," she explains, "You need someone that understands the games and can make true analysis on the games."

"One thing I don't like is how people tend to overcomplicate things - your tools should be as simple as you need them to be, no more complicated than that"

Alessandro Scoccia Pappagallo, Google

But if you do have that person who understands, can make the analysis, and has the time and drive needed to make their own tools, your analytics division can become a huge part of how the company operates. Best of all, it doesn't even need to be complex, as Pappagallo explains: "You can do so much stuff with very basic stuff - another thing I often see is data scientists or statisticians hold a grudge against Excel. I think Excel is amazing! It's not free, but something like Google Spreadsheets is, and you can do amazing stuff with it. You can write powerful tools - I've seen people do crazy stuff with Excel. Stuff I don't understand. And it's a tool everyone has a grasp of.

"One thing I don't like is how people tend to overcomplicate things - your tools should be as simple as you need them to be, no more complicated than that. If whatever problem you have can be solved with Excel, use Excel. Why would you write code? I want to write the least amount of code I can. I love writing it, but having to write it just because it's functional and not because I want to." What it all boils down to, though, is this: should you, working in a smaller development team, actually bother with analytics in the first place? Darley's opinion is a resounding 'yes': "I think it's essential. If I just look at the scale of improvements we're able to make at all stages of their life cycle by measuring stuff, carefully doing experiments, looking at what works and rolling that out, it's something that benefits all companies of all scale, I'd say."

Coming from a man at a studio like King, with its very successful projects in the Candy Crush world, the words carry weight - and Darley is under no illusions as to how much impact analytics has had on the growth of King: "We're a very data-driven company," he says, "So it would be hard to envisage King without a data-driven approach... what might the company look like? I would certainly say there have been some key moments in our history where having a good understanding of the data has been instrumental in strategic business decisions that were made, for example our migration from skill games to Facebook games and onto mobile games today. At the very least King would look very different if we had not been looking carefully at the data along the way."

It's also something that Bayley, coming at analytics from the perspective of the smaller, just-starting-out perspective, is positive about - if you can, at least: "I'd say if you have any sort of resources to be able to look into the analytics side it's worth it," she explains, "I've been lucky in my company that each of the studios is really open to data because it's something they didn't have before. Every time I sit down and have a first presentation with one of them, they're very grateful, they have no idea, it helps them look at things in a different way. If you can't invest any time at all, there's still free tools that make it a lot easier to at least look at the basics. I think it's worthwhile no matter what."

"If you don't hire a data scientist, you could hire someone else, you could use the money for a marketing campaign or something. Adding a data scientist who's only there to make Excel documents or create a database is a huge waste of resources"

But there is always the question of money. Does it always make sense to hire a data analyst for your studio, even if you're on the smaller end of the scale? No, says Pappagallo, who admits the question is one he thinks about a lot: "I can see cases where hiring a data scientist is not the best option, simply because when you're talking about doing something you always have to think about the costs of doing it versus the costs of doing something else," he says, "If you don't hire a data scientist, you could hire someone else, you could use the money for a marketing campaign or something. Adding a data scientist who's only there to make Excel documents or create a database is a huge waste of resources - you're going to make the person feel like they aren't doing anything exciting, and at the same time you're just wasting money. You should never do that.

"Either you don't hire a data scientist and you continue to carry on with what you're doing - if you're lucky and have a good idea it may work... I'm hearing more these days that people think they can't do anything without a data scientist, which is madness. People were doing great projects before data scientists came along, you can still do it now. If you do go with hiring one, you should make sure you think about them as not just a node for a network of data, but as someone who can educate others inside your company about what they do, who can build tools for people. For small companies that would make more sense, someone who's more between a data scientist and an engineer - someone who can take care of data and your infrastructure, building tools for you to use and so on."

So the solution seems to be, if you have the funds, to hire someone who fits two or three roles and can pass on the skills to other members of your team - to build a strong foundation of analytics when you're still a small studio, and grow things accordingly over the months and years that follow. Sometimes, as Pappagallo admits, it's not worth hiring someone to handle analytics at all: "If you don't think, for example, that you have the ability to implement all the changes that the data scientist has tabled, for whatever reason - maybe your backlog is so huge you'll never reach the end of it - then maybe you don't really need a data scientist.... If you don't have time or flexibility to be able to implement changes, why bother checking which one is better?"

But, generally speaking, this is an area in which more and more studios are looking to help to grow - to help understand their audiences and build games according to the needs and desires of the paying and playing public. You shouldn't run out and hire the first person you find without making some big decisions, but you should remember: there are options, and it's easier than ever to get that data flowing.