Tech Focus: The battle against latency

So what do the results typically tell us about latency in console games? Well, there is a clear link between frame-rate and controller response. For 60FPS titles, in optimum conditions we typically see response at four frames, or 66ms, with titles such as Call of Duty 4, Guitar Hero and Ridge Racer 7 offering this exceptional level of response. However, games such as WipEout saw an increase to 83ms, while our experiments with LEGO Batman saw the game respond fairly sluggishly at well over 100ms.

For games that operate in the 30FPS range, latencies increase significantly. Up until recently, it was impossible to find any game that offered a sub-100ms response. Several titles, such as Unreal Tournament III hit 100ms, but several other Unreal Engine 3 games such as Mirror's Edge and Bulletstorm offered a 133ms latency between button press and on-screen response.

It's interesting to note that Bulletstorm received no criticism that we were aware of for its controller response, and yet Guerrilla Games' Killzone 2 did. We measured this with a baseline latency of 150ms - just one frame more than Bulletstorm. The question here is intriguing: is there a threshold at which latency becomes noticeable and detrimental to the gameplay experience?

Going forward, hammering down latency is going to become increasingly important if developers are looking for the best performance when readying a game for a service like OnLive.

It's worth bearing in mind that these measurements do not include the additional lag of the display. In the case of Killzone 2, married up to our Dell 2405FPW with its own three-frame lag at 720p, we would be seeing a minimum of 200ms - a fifth of a second. The issue with Killzone 2 is that the 150ms is just the baseline. Frame-rate drops can add two or three more frames to the delay, and it is perhaps this inconsistency in response that is one of the major problems: players cannot adjust to the controls because the level of response is not so consistent. Guerrilla Games patched the title, but the only real adjustments seemed to be in the "dead zones" of the analogue sticks rather than addressing the level of response itself.

Things changed radically with the release of Killzone 3, with our measurements indicating that the baseline 150ms latency had dropped down to 116ms - a substantial achievement.

"There were a lot of changes to all systems related to input, movement, gunplay and player response," Guerrilla's technical director Michiel van der Leeuw told us during an in-depth GamesIndustry.biz inquisition on the development of Killzone 3.

"We fixed a few bugs where we inadvertently added latency, but we also tightened up deadzones and the responsiveness of particle effects to show your bullet hit something. Anything that wouldn't change the weight, but improve the feel and control of the player we polished up."

The high-speed camera technique was also utilised by the tech team in helping to reduce unwanted lag.

"We did a lot of testing with high-speed cameras to verify our changes and see if we were still on track. We also added a system to the engine that would track an event through the engine from the initial button press as it would travel through code, all the way down to when the RSX would send it to the TV," van der Leeuw added.

"So you can see when the button was pressed, when the game code responded to it, when the bullet was fired, when the particle was spawned and so on. If the high-speed camera then indicated we had long latency or jitter, we could pin-point the exact problem."

Criterion's superb, multi-award winning Need for Speed: Hot Pursuit managed to cut latencies to what we believe is an all-time low for a state-of-the-art current generation console game. Having worked in the 60Hz realm with all its previous Burnout titles, the developers were keen to make their first 30FPS game as responsive as possible, and this required an extensive reworking of the engine, binning the tradition approach of running game code and rendering on different threads and instead utilising a single thread instead.

The result is quite extraordinary. Criterion's last major game, Burnout Paradise, ran at 60FPS and offered a 66ms input lag. Need for Speed: Hot Pursuit, capped at 30FPS offers an 83ms response: just one additional frame of latency for a game running at half the frame-rate of its predecessor. The PC version was faster still, measured at an incredible 50ms of end-to-end lag.

"The way the architecture works is to run the game simulation internally at 60FPS, and it's polling the controller once for every simulation step so you get as up-to-date inputs as possible," Criterion tech mastermind, Alex Fry told us.

"The render code (building display lists for the GPU to consume the next frame) immediately follows the two controller poll/simulate loops, and then it waits for v-sync. Thus on CPU we get two 60FPS updates and one 30Hz render in a total of 33ms. When the CPU is done and v-sync is hit, the GPU kicks off and renders the scene while the next simulate/render frame happens in parallel on CPU. Once done the GPU also waits for v-sync (which also syncs it to the CPU), thus adding another 33ms. Then TV scan-out happens to get the final image to screen, which adds a final 16.6ms. On the PC version, it's obviously able to go to 60FPS so we don't clamp it to two simulate loops per render."

Going forward, hammering down latency is going to become increasingly important if developers are looking for the best performance when readying a game for a service like OnLive. The lower the baseline latency, the lower the impact of additional, completely unavoidable lag caused by video encoding, transmission of data and video decoding client-side. But just on a general level, the tighter controller response, typically the better the game experience becomes...