"Retention is not the same as happiness"

Laralyn McWilliams, the former creative lead on SOE's Free Realms, has warned free-to-play developers away from using metrics as the basis for the roadmap of their games. Data can indicate a great deal about a game, she argued, but it falls short in the area that matters most.

Relative to the growth of the free-to-play business model outside of Asian markets, McWilliams is something of an old hand. As creative lead on Free Realms, she was instrumental in the creation of what was, at that time, one of the most ambitious free-to-play games ever attempted in the West.

However, after nearly a decade of thinking about and working with the free-to-play model, she has reached certain conclusions about the dominance of metrics in the way games are designed. Fundamentally, metrics cannot indicate the player's emotional attachment to the game, and in some cases they can be misleading.

"There's no measuring spoon for love. You can't quantify it," she said in a session at Casual Connect Europe in Amsterdam. "Retention is not the same as happiness."

In all media, the consumer often continues to engage long after their satisfaction with the experience has started to wane. People watch a TV show even when they're bored with its progress, they read books in a series that has long since peaked, and they play games even when they have started to lose interest in - and even resent - the experience.

For McWilliams, metrics have failed to offer a clear picture of when this process starts to happen, and the motivations that set it in motion. She used A/B tests as an example of a method that seems to be effective, but is often misleading due to the complex and dynamic nature of online games. A/B tests do not take place in a controlled environment, yet major decisions about a game's design are directly informed by their results.

"Really, after a few months in your game, every player is a Frankenstein of all the A/B tests and updates you've ever done," she said. "The decisions you make from an A/B test are not necessarily good decisions for the game as a whole."

According to McWilliams, the sort of granular detail metrics offer can be seductive, granting a sense of control and confidence. But the reality of operating a game in a live environment is akin to climbing a high rock-face. The path to the top is impossible to plan from the ground. You can only reach out and try to find the next handhold, and strong instincts can be just as useful as knowledge of the topography.

"The point is that It's a mix of logic and emotion that goes into our decisions as game designers," she said. "That's why we can never design by metrics."

McWilliams did not pretend to have the definitive answer to making the player happier. It is, she maintained, very much an open question, and one worthy of detailed discussion.

However, she was more decisive on one matter. Pulling up a slide of a chart depicting the average player lifespan on mobile platforms - descending rapidly with the passage of time - she said, "I maintain that this is crazy bullshit." She pointed to EverQuest as an example of a game that managed to build a lasting relationship with its players, and without the high churn rate that is basically expected by so many developers.

Speaking in the subsequent Q&A session, Teut Weidemann, senior online game superviser at Ubisoft, agreed wholeheartedly with McWilliams' stance.

"People forget that we're in the online games business," he said. "People see the metrics and only think about the monetisation. At Ubisoft we call players 'fans'. It makes you think differently."