Go Play Inside - Is Project Tango Going to Underpin AR?

At the far end of the expo hall at Unity's Boston event last month, there was a colourful booth with a big queue. At the front of the queue, eager developers were handed a jury-rigged Nerf rifle with a small-format tablet attached on top, displaying an augmented view of the wall in front of it. Essentially a high-tech version of a fairground duck-shoot, it took a little closer inspection to understand why this attraction was pulling in such big crowds. Firstly, it was Google, demoing as yet unreleased technology. Secondly, that technology was capable of far more interesting feats than simply replicating the crooked air-rifles at the county show. Thirdly, and possibly most importantly, they were giving away free dev-kits.

The Project Tango dev-kit, which you can see below, is a Tegra-powered tablet which is sleek but, on the surface at least, initially unremarkable. However, it's what's inside that counts - specifically the device's array of cameras and sensors.

Featuring a 4MP 2µm pixel camera, a motion tracking camera and an integrated depth sensor, Project Tango is able to accurately map an environment in real time, creating a 3D mesh of its surroundings on the fly. It sees surfaces, distinguishes between objects and accurately measures distances, building a full picture of everything it sees. Whilst many of the applications for it are likely to fall well outside the remit of the gaming world, Google recognises the power of games to capture the imagination of the public so, much like with VR, it's our industry which will likely form the tip of the spear as the project worms its way into the greater consciousness.

"The goal of project tango is to give our electronic devices, our computing devices and ultimately our wearable devices, a human scale understanding of our physical space and motion"

With over 4,000 dev kits in the wild, there are already a few games submissions available which showcase the location and environment based potential of the tech, but Google's presence at Unite was aimed at the proliferation of games to show off to new users and potential customers. During an hour long presentation in a packed hall, Tango's project lead Johnny Lee gave an extensive and enthusiastic live demonstration of many of the device's capabilities, showing how it could create shooting galleries with targets popping up from behind furniture, or build 3D meshes of whatever it saw even at running pace.

Clearly AR could be a big winner from this sort of tool, but what else does Google have in mind for Tango's gaming potential? GamesIndustry.biz sat down with Lee afterwards to find out.

GamesIndustry.biz: Tell us a little bit about your background and the origins of the project - was this something which came from working with Google Glass?

Johnny Lee: "My personal background is that I've always had a strong interest in interface technologies, input and output. 3D input has always fascinated me. I did actually join Google during the time that Glass was being developed but that product was always more of a notification device. The computer vision techniques we use for our 3D vision processing require pretty aggressive sensors as well as computing power, and that just wasn't available in that form factor.

"So when Google acquired Motorola I decided to head over there and see if I could start a project based around 3D motion sensing on mobile devices and that's where we started in the advanced technology and projects group. A few years later we moved back into Google and now we've matured into our own product group."

GamesIndustry.biz: Is this a game focused technology, or is that just the thin end of the wedge?

"The grand vision that I'm excited about is the next generation of GPS. In my opinion GPS has had a transformative effect on the way that both society and industries operate"

Johnny Lee: "The grand vision that I'm excited about is the next generation of GPS. In my opinion GPS has had a transformative effect on the way that both society and industries operate. If we're able to solve the problem of accurate positioning, regardless of what beacons or satellites are around, there's a number of new things that we can expect our devices to be able to do when they understand that we're in this room in the same way that I understand we're in this room.

"So the tablet and phone form factors are the ones where the majority of the industry are still manufacturing devices and we see indoor navigation as a major use case which we can enable. Indoor navigation is very exciting for some people, but it doesn't make you jump out of your couch and say 'I'm going to go and buy a new device!' But the tech does enable a lot of other things and games is one of the things that we think is more appealing to consumers. There's also more utility capabilities such as measuring the distance to walls or mapping houses. So there's a strong set of professional use cases, but Google is at heart a consumer company so if we can get broad adoption then that's a success for the project."

GamesIndustry.biz: You mentioned that some of the tech involved isn't dissimillar to that used by Kinect. Is there any shared pedigree or staff?

Johnny Lee: "No actually! I still chat to some of my old colleagues there, many of whom are now working on Hololens, but the core technology and the component vendors are Google colleagues."

GamesIndustry.biz: I'm presuming that this is going to be Android specific, at least for now...

Johnny Lee: "For the time being - simply because Android is such a large platform for Google, but we're actually very excited about seeing this technology being used in AR, VR, Robotics. Those domains are so new that whether or not to use Android isn't exactly my call."

GamesIndustry.biz: You've got 4,000 kits in the wild now, are you starting to see commercial use-case submissions?

Johnny Lee: "I think as the eco-system matures we see both more mature content and a stronger interest from manufacturers. Coming to [an event] like Unite is one of the ways we can insert some energy into that because we can show the output of our partners here and then OEMs will be more excited to build more projects in volume."

"I mostly get excited about new genres of content. Content that understands your environment is just one that hasn't been created yet"

GamesIndustry.biz: When thinking of how games can take advantage of this, what sprang immediately to mind was Ingress, but what are the advantages of this for game devs who make games for the more mundane mobile gaming circumstances: the bus, the sofa, the toilet?

Johnny Lee: "For better or for worse, I think the bus, the sofa and the toilet are pretty well covered! I mostly get excited about new genres of content. Content that understands your environment is just one that hasn't been created yet. In VR people talk about things needing to be metrically accurate - you have to make sure that inter-pupillary distance is accurate, hand controllers....So there's a new genre of content that we're creating for VR.

"But even that broadly assumes that you're staying in one place. You're either sitting in front of one computer or a fixed room. Tango devices allow you to make games that understand the whole world, from your living room to your office building to the whole city. That has a variety of new game design challenges that are, to some degree, form factor agnostic. Whether or not you're on a handset or a tablet or a headset, you need to make a story that works in any space possible. I think that's probably the most interesting aspect of it, from a game design standpoint.

"Phones and tablet are extremely popular today, so those are the form factors we're focusing on for now."

GamesIndustry.biz: Do you see this remaining a feature of general use devices, or becoming a specific product?

Johnny Lee: "I don't really know, that's sort of a consumer product question: do people want to buy a dedicated headset or do they prefer investing in these general purpose phones? The future suggests that people tend to gravitate towards more and more specialised devices. I have more devices now than I did ten years ago, so I see that trend continuing.

"For the core technologies of Project Tango I'd like to see us modularise the work that we've done into something like a GPS module, by which I mean that so many devices have GPS tech in them now, I can imagine that being the case for Tango in the future."