More data, "real feedback" would greatly benefit indies - Clever Endeavour CEO

Advice is a tricky thing. Just ask Clever Endeavour CEO Richard Atlas. GamesIndustry.biz caught up with Atlas at the Montreal International Games Summit last month.

"The first step when people are giving advice is to be aware of the amount of bias that's in that advice," Atlas said. "I have one game that succeeded. That is survivorship bias defined, so it has to be taken with a grain of salt. But I can give my opinion on things and say I think this would be good for your game, but it doesn't make my opinion that much better than the next person's."

Watch on YouTube

That game, Ultimate Chicken Horse, began as a game jam experiment in late 2014, a test run embarked on by Atlas and the rest of the founding Clever Endeavour team to see if they could work well together in a high stress situation. The game jam effort was never supposed to be a debut title, but they received encouraging feedback, which prompted them to work on it more and release more advanced builds, which garnered yet more positive feedback until it became clear that the project, a party platformer, could have a commercial future. It was released in 2016, and received an Independent Games Festival nomination in the design category in 2017. Clever Endeavour released Xbox One and PS4 versions last month, and is now working to bring the game to the Switch this year.

Prior to that game jam, Atlas said he had virtually no presence in the industry. He'd been thinking about game development and design for a few years at that point, but didn't have any sort of connection to his local Montreal dev scene.

"The first challenge was just getting in, socially," Atlas said. "We come to these events, and the industry's quite small. It's big [in Montreal] compared to other places, but everyone knows everyone. It's a pretty tight-knit community. So when you come into these events, it seems everyone knows everyone and you're this outsider, especially having nothing to show, no playable game to say, 'Hey look I'm doing this thing.'"

"It's hard to get an understanding from someone telling you, 'I did this thing in my game and it worked really well.' That's great, but it's just a story"

However, that originally intimidating scene proved itself more welcoming than anticipated, helping to convince Atlas that getting into game development was the right move.

"Feedback from people in the industry was really important, the fact that people were so open and accepting even though you didn't have anything to show. They were sharing techniques and management stuff and all sorts of information I would never imagine people sharing in other industries."

These days, Atlas sees a different set of challenges facing Clever Endeavour. The first one to come to mind was a lack of data.

"In a lot of other industries - in tech and so on - you know exactly who your userbase is, the demographics of your users, a lot of stuff," Atlas said. "You can put marketing money into something and see direct results. In the indie game scene on PC and console, there's really no data. It's hard to get an understanding from someone telling you, 'I did this thing in my game and it worked really well.' That's great, but it's just a story.

Even eight people who did a certain thing in their game and it worked is still an extremely small sample size. So I think there's a lot of people giving a lot of advice based on stuff that is not scientific, but we're giving it in a way that is saying, 'You have to do this.'"

But even if advice from other developers isn't scientifically reliable, Atlas still thinks you need lots of it, as he explained when talking about the second big challenge Clever Endeavour faces.

"I don't think the visibility thing is the biggest issue. I think it's the quality of games, and that's not improving because we're not asking our peers enough to look at our stuff"

"Visibility is the obvious answer," Atlas said. "I almost don't want to say this because it might seem survivorship bias-y, but I think there are ways to get your game in front of people. I don't think it's that difficult. Getting feedback from other developers, getting feedback from the community, and actually understanding what's working or not working in your game is the thing that blocks further visibility. I don't think the visibility thing is the biggest issue. I think it's the quality of games, and that's not improving because we're not asking our peers enough to look at our stuff."

Atlas said it's important to get feedback early both from developers as well as the public, "real feedback that's also in your face, like no bullshit, not saying 'Oh yeah, sure, it's fun!' and then not being interested." He acknowledged that might run a bit counter to the warmth and encouragement that helped convince him to become a developer in the first place, but said he's working on setting up a feedback circle for developers looking for "in your face feedback that's no holds barred," and willing to give the same in return.

"So there's no sensitivity," Atlas said. "There's no dancing around being nice to the person. It's really constructive criticism and feedback of your game. I think we do a decent job of that if you look for it, but a lot of people don't. They have this idea of, 'This is my vision' and they know how great it's going to be, then they ignore the feedback they get or don't look for the feedback. Then at the end of the day, they have this thing that's missing some key elements and they don't realize why."

Whatever Clever Endeavour does as a follow-up to Ultimate Chicken Horse, expect the game to be made available for such feedback early and often. Atlas said taking the game to demo nights and industry events on a regular basis almost forced the team to face such feedback frequently, giving the team new perspectives Atlas said were "super important" for the development of the game.

Even though the Clever Endeavour team has vastly expanded its experience in game development through the development of Ultimate Chicken Horse, Atlas isn't assuming that will produce a better selling title.

"We definitely don't expect the next game to do as well because I think it's unrealistic to think every game you make is going to sell that well," Atlas said. "If it does, great. But because we made some money off this game, we don't have as much pressure on the next one."

Ultimate Chicken Horse has probably given the team some runway before it needs another hit, but that's not a luxury Atlas wants to take advantage of.

"We're making commercial products," Atlas said. "I don't see us making something super experimental that's not likely to be able to sell. But we're not necessarily saying, 'We have some of this multiplayer party game market. Now we have to make another multiplayer party game.' That's definitely not the mentality. It's more like, let's prototype some stuff, figure out what we think is fun and good and interesting, what we think can help us progress as individual programmers, artists, and designers, and what we think will also be able to be commercially successful."

Disclosure: MIGS has a media partnership with GamesIndustry.biz, and paid for our travel and accommodation during the event.