MMO Week: The Trouble with Trials

In the past few years it's become noticeable that the concept of an MMO beta test has changed significantly. Once upon a time they were there for dedicated souls, who could exchange the honour of being among the first to get a glimpse of a new game in return for testing the content, reporting bugs, and not judging the book based on its draft cover.

Developers could rely on hardy gamers to look beyond the torn visuals, choppy textures, low frame-rates and broken crafting modules to see what they were trying to achieve - what potential the game had to excite, entertain and engross for many a long subscription month ahead.

But now it seems like the beta test of any new MMO simply exists for people to jump in, judge what they see as effectively a final product, spread the word that there are torn visuals, choppy textures, low frame-rates and broken crafting modules - usually in the game's own community forums - and walk away.

Of course, that's not true of all beta testers, but it's a worrying enough trend that nowadays a game has to be on the very cusp of being complete for anybody to sign off on any kind of preview for large numbers of the public to gain access. Beta tests have - in a sense - become live demonstrations - but why?

There are a number of likely reasons, the most obvious of which is the need for companies to avoid a high profile server crisis upon launch, because after all, there's nothing quite like a person's inability to gain access to a game on launch day to persuade them not to come back after the first 30 days.

But actually, is that really true? Certainly there exists a need to stress-test servers, to find out the optimum numbers of players per server, to see what happens when too many people gather in one place at the same time, and to find out how the various modules within the game - an auction house, perhaps - would cope when it's being used to the full.

Blizzard's biggest problem with World of Warcraft came in the first weeks after launch, when the game was so inundated with demand from players that the servers couldn't cope, and the company had to open more of them - a move that took some time to complete effectively.

Those initial problems gained a lot of publicity, but such was the quality of the game that people kept coming anyway - although that's a special case, and arguably with most other MMO titles a similar performance would spell serious trouble. So companies will now opt for server stress tests and the admission of tens of thousands of people into a game before it's necessarily complete.

Of course, the other problem is that for an MMO, there is so much content to test that an in-house team generally isn't enough, and hundreds - if not thousands - of beta testers can actually be very effective in spotting the myriad of eventualities that large persistent worlds can supply...but that's assuming that they actually bother to report them.

As a result two phases of beta test have been introduced - closed beta and open beta, the first for the more traditional testing, the second as more of a live game demo, a move that's good in theory, but for plenty of games I've seen the open beta is still unpolished enough to generate significant concern in the minds of gamers.

One of the biggest benefits to the community of an MMO is that it's a game that can be constantly developed, updated and evolved - it's one of the attributes that justifies the monthly subscription fee, after all.

But too many companies are using that as an excuse to launch games that simply aren't ready, and another increasingly worrying trend is the amount of bug-fixing that goes on during the first 30 days after launch - the so-called 'free' trial period that most games ship with.

First of all, this time isn't really free, because somebody who has just paid the initial fee to buy a game should really expect to be able to play the game for an initial period, so the concept of initial play time for free is an irritating misnomer.

Secondly, although an MMO is expected to have a lengthy lifespan, the vast majority of a game's publicity will - at the moment - come at launch, and if the game suffers from significant bugs, or modules are missing, or content is missing...that's a stigma that is very unlikely to go away any time soon.

Sadly there have been a number of examples of titles suffering from a poor launch experience. Last year Vanguard, probably held up as the current forum buzzword for MMO failure, was released too early and while the title's marketing, background and subject matter were all good, the user experience was poor - and the game, despite changing a lot since launch, hasn't yet managed to shake off that initial reputation hit.

This year, Pirates of the Burning Sea saw a great idea and generally solid execution in terms of game design suffer horribly because of technical issues, broken quests, an absence of polish - in short things that should have been nailed in the last few weeks of testing before the game was released. Recently a number of servers were shut down, and a title that should have added an extra dimension to the global MMO offering is suffering.

As a long-time and dedicated MMO player, it often disappoints me to see a game nearing release that still suffers from problems that are generally cosmetic in nature, but which are unlikely to be fixed before launch, because the developers want to nail the bigger picture things right first - not because I don't think they are problems that can be fixed quickly, or because it disturbs my own play/test experience, but because it's only a matter of time before a forum thread about them will start to influence people's opinions.

Unfortunately, although the physics in the world may work and the shadows and textures look great, if there's anything else that's not finished it will show up like a sore thumb - and compare badly to the MMO that a person is already playing. Worse still, chances are that comparison game will be World of Warcraft.

Sure, if only people were more dedicated in their beta testing, or more realistic in their assessments and expectations, perhaps launch periods would be a happier, more polished place - but the reality is something we all have to live with, and reliance on beta for anything significant is a dangerous step.

While launching an MMO is unquestionably a huge task, and while there are often serious commercial pressures in play pressing for revenue to start coming back in, if a game is ever to actually rival Blizzard's behemoth for press attention and player subscriptions, everything has to be right - at launch certainly, but arguably in open beta as well.

If it's not, you won't find the player base is as forgiving and dedicated as it was back in the early days, when people played MMOs despite a user interface, not because of one. Players have moved on, and for the most part the industry still has a little catching up to do.