Niantic: "AR is not just visual"

The next frontier of AR on mobile will be audio, according to Niantic Inc. CTO Phil Keslin, because sound offers a solution to the more socially awkward aspects of playing an AR game.

Speaking at Techcrunch Disrupt in San Francisco yesterday, Keslin pointed out that people won't be comfortable staring at a smartphone screen while walking around. "It's very unnatural," he said. "It makes them look like a total doofus if they're doing it for an extended period of time."

Niantic's last game, the enormously successful Pokémon Go, limited the amount of time users had to actually look at the phone's screen. According to Keslin, "the only time they really use it is to share their encounter with the Pokémon. To take that one picture, which is natural... Everybody takes a picture, and then they're done. It's not walking around the world with the phone in front of their face."

Audio offers a more natural alternative, with Keslin highlighting that listening to audio through earbud headphones is a form of augmentation, and one that a huge number of people experience every day. Niantic experimented with many audio features for Ingress, in fact, though not all of them made it into the shipped game.

However, audio will play a key role in Niantic's next game. Keslin said that the studio was not "a one-game wonder," but declined to offer further details on the product or when it would be released.

"I think audio is significant," he said. "It's one of our senses. It's one of things that really drives us. I want to look at ways to incorporate audio in future titles. AR is not just visual."

The idea that AR should be more than visual is becoming a theme for Niantic. In the aftermath of unveiling of the iPhone X, CEO John Hanke questioned the focus that Apple appears to be placing on the visual side of AR.

"Apps that merely place a digital object on your kitchen table don't really qualify as 'AR' in our view," he said. "Even when used out in the world in the 'right way,' AR suffers from a challenging form factor when accessed via a phone. Holding a phone in front of you to align an AR view is, honestly, a little awkward.

"Based on experiences with apps that are mostly focused on this visual aspect of AR, some will conclude that AR is a gimmick that lacks real utility. That's a bummer, because it really is the first step to something that is going to transform the world as we know it."