Crafting believable VR characters in Blood & Truth

You're holding someone at gunpoint. The man you have been chasing through a gangster-packed hotel whimpers in front of you, pleading for mercy. Floating dialogue choices enable you to choose whether to give it to him, or to threaten him further. Eventually he tells you what you want to know.

Or you can just shoot him in the leg.

Instantly the scripted conversation stops. He drops to the floor, clutching his injury and writhing in pain. He calls you a bastard, and tells you what you want to know.

This is the example Stuart Whyte, director of VR product development at Sony's London Studio, offered to attendees at this month's Develop:VR conference. It's a scene from upcoming title Blood & Truth, a PSVR-exclusive shooter set in the UK capital's underworld and was easily the biggest draw at the event's expo.

After showing the two examples of this exchange, Whyte stressed how important actors will be in the future of virtual reality, making conversations with characters much more believable and therefore more immersive. GamesIndustry.biz caught up with him following his talk to find out more.

The value of interactive scenes like the one shown from Blood & Truth stems from Sony London's previous title, PlayStation VR Worlds. The new virtual reality shooter is heavily influenced by London Heist, the most played section of the PSVR launch title, although Whyte says the team benefitted from each of the games bundled into this product.

"VR Worlds, for us, was fantastic," he tells GamesIndustry.biz. "It was five different experiences that really helped the team get to grips with what VR meant. We learned a lot of really valuable lessons on that, and with Blood & Truth it's about taking those learnings forward."

The idea of conversations that react to player actions stemmed from a number of scenes in London Heist. When players aren't gunning down the game's enemies, they're often sitting through an exchange with the game's various characters - from a briefing in a pub, to being locked in with a man threatening you with a blowtorch. Whyte observes that these scenes were "really powerful but quite passive."

"As soon as people are given a playset, they start to experiment with it"

"As soon as you get to the stage where you can have some dialogue choice, it adds to the flavour, the immersion and the sense of presence," he says.

Observing how players interacted with these scenes also gave the Sony London team ideas for further interaction. An early scene set in a pub, for example, allows players to pick up a lighter and use it to light a cigar. This was based on what players were trying to do while testing earlier builds, and Whyte predicts scenes will become increasingly interactive as VR matures.

"As soon as people are given a playset, they start to experiment with it," he says. "You get the same in normal video games.

"You used to shoot at a wall and there would be no damage on it at all. Then we got to the point where there were decals on the wall that showed where you had shot, then you started getting games where there was real damage on the wall.

"It's going to be the same with VR. It's going to get degrees and degrees better all the time, and that's what makes it super awesome to be in this space. Where it's at now is so exhilarating from an immersion point of view, but looking at that long-term vision of 15 to 20 years in the future, it's going to be crazy."

Blood & Truth's interrogation scene is a degree up from London Heist, but one still limited by current technology. The perception of player agency is a little bit smoke and mirrors - there are, of course, limits to how the player can influence events unfolding around them. In the case of Whyte's interrogation example, it's a fairly safe bet someone would attempt to shoot the other character, so programming in a reaction is more of a scripted branch than a truly organic moment.

But there are only so many player actions developers can predict and account for. Eventually AI will advance to the point where NPCs can react to anything the VR user could possibly attempt, but until then we are limited to how many choices developers can or will allow.

This is why Whyte stresses the need for quality performances and believable characters. If the scene is immersive enough that it feels like it is reacting to the player, they will be less aware of how preordained events actually are.

The best way to ensure a character is as believable as possible is to use real-life actors. The whimpering coward Blood & Truth's hero encounters has been fully performance captured, his likeness based on the real actor, his voice provided by the same man. It's a far cry from the much simpler requirements of the voice acting found in most games.

"We used to casting our characters based on whether they sound right... Now we focus on have they got the right voice, can they act and do they look the part as well?"

"[Developers are] used to casting our characters based on whether they sound right," Whyte explains. "The voice and the accent has to be right, of course, but we're putting them in mo-cap so they have to be able to physically act. It's almost like theatre acting because it's quite close up. It's not like TV acting, where you can do cuts or whatever.

"And then there's the look. They might have a great voice and can act really well, but if they don't actually look like our vision of the characters, we need to go to someone else. Our audition process we've been going through has really been focusing on that: have they got the right voice, can they act and do they look the part as well?"

Sony London is currently in the middle of casting and unable to discuss more about who will be involved in the final product - the examples shown are more than likely placeholders, albeit convincing ones. But Whyte does reveal a little about the type of actor that the studio has found fits best within a virtual reality project.

"A lot of the cast have TV and film background, but a lot of them have stage backgrounds," he says. "Being in VR, and being in an environment with an actor... [the player is] right next to the person acting. It's really close, and the closest you can get in real-life is one of those stage adaptations where you're actually in the environment. It's that level of acting out [that we need] rather than acting to a camera, or whatever."

Finding the right actors and even giving them the best look has required new skills that are unusual for a games development studio - even one on the cutting edge of a technology like virtual reality. For example, Whyte declares that Blood & Truth is the first project he has ever worked on that includes a costume buyer. One member of the team, an experienced costume buyer, purchases the outfits worn by each of the characters within the game.

Sony London has even found companies on the internet that sell "high-tech helmets and stuff", which the studio has licensed and scanned in for the high-end military players will be facing towards the end of the game. The actors will also wear this tech, where the mo-cap equipment allows, to help immerse them in the scene as they perform.

Of course, not every VR studio has the budget and backing of a platform holder and is therefore unlikely to be able to afford motion capture, stage-trained actors or even a costume designer. But there are still techniques they can use to make scenes more believable.

"We're only a year into the lifecycle of PSVR and we are just one of the many developers trying to define that language, working out what works and what doesn't"

An example Whyte gave during his talk was 3D storyboarding, a technique Sony London has developed since working on VR Worlds. Put simply, the firm maps out an environment and 'blocks' scenes - much like theatre, this essentially maps out where characters will be when they speak certain lines or how they move around during a scene. Whyte even showed off an example, where the VR user finds themselves within a 3D storyboard scene, complete with cardboard cutouts to represent characters - not unlike what you would see on a typically Hollywood storyboard.

"3D storyboarding is not an expensive thing to do," Whyte says. "It's something I hope teams of all sizes can use."

Of course developers need more than believable characters in order to help complete the illusion of full interactivity. Realistic 3D audio, physics and plenty of object interaction also go a long way to convincing the player they're in another space.

"It's all the little nuanced things that when you're in VR and you pick up a phone and put it down, or you look at a character's face and they react to you... It all gets you to the point where you ultimately forget you're playing a game and your sense of presence is such that you're right in that moment. You feel like you're there."

By way of example, Whyte shares with us some audio trickery the Blood & Truth team has been working on.

"Ray tracing in 3D graphics bounces light around a room and that helps you get really photorealistic graphics," he says. "We're doing a very similar thing with audio. All of our materials in the world are tagged with what type of material they are, so that when you shoot a gun at them they sound different - wood sounds different to metal, and so on.

"We have a doctor of acoustics on the team who has made it so that when a character is talking as he walks around an environment, we actually bounce the soundwaves that come from his mouth off the different materials. If you're in a very cold room, acoustically, you'll actually hear the difference. It's little things like that which feel really nuanced but really add to the experience.

"This is the exciting thing about VR. The design language and the rules of how to do it... We're only a year into the lifecycle of PlayStation VR and we are just one of the many developers trying to define that language, working out what works and what doesn't."

Whyte was one of the four VR experts who joined us for the latest episode of The GamesIndustry.biz Podcast. You can listen to more of what he has to say about virtual reality development right here.