Nimble Sense: Hands on with virtual reality

Corner a virtual reality sceptic and there's one thing they'll point to over and over again: how do I interact with that reality? Answers range from simple controllers to complicated treadmills that will look really weird in your lounge, but one San Francisco company has a better answer, and you already own them. Your hands.

Think Kinect, but better. Think super sensitive gloves, but without having to wear the actual gloves. The Nimble Sense is a surprisingly small camera that clips to the Oculus Rift and scans your hands, allowing you to manipulate objects and make gestures in the virtual world.

"The Oculus Rift brought people's eyes into VR, that's great, but we also want to bring the rest of their bodies into VR, starting with the hands," explains Robert Wang, one of the creators of Nimble Sense.

"We want to be part of this journey that people are trying to figure out; how is this all going to work out?"

Instead of seeing colour the camera sees depth, taking a scan of the whole field of view every 20 milliseconds. That data is then sent to the Oculus Rift, for each eye. In practice it's quick, with no noticeable latency issues for the untrained eye. I'm able to push buttons in a floating interface to start and stop films and, in another demo, shoot lightning from my palms. Wang shows me more, a bomb defusing demo that could have been from any first-person game and a simple demo where he manipulates a cup and die.

It's currently on Kickstarter with 22 days to go but has already beaten its goal, raising $104,653, some distance from the original $62,500 request. Scan the backers list and you'll even spot Oculus VR CEO Brendan Iribe on there.

"Kickstarter is about raising money and it's also about tapping into the community. So every developer for the Oculus Rift probably backed Oculus Rift for their Kickstarter; they already have Kickstarter accounts. We can actually tap into those people and say if you're interested in building new interactions, if you're interested in trying out some crazy new hardware, another DK1 in some ways for tracking hands, then get in touch with us."

"This is not the ultimate solution yet, it's a part of it"

Wang and the team wanted to use Kickstarter to gauge feedback and build a community, to tap into the collaborative way of working that seems so unique to the current virtual reality development space.

"We didn't believe in VR for a long time, we had to sort of be dragged into VR," he admits.

"It's only after we started going to some of the meetups, like the Silicon Valley Virtual Reality Meetup, that we were really convinced. You could feel the level of excitement, and how quickly everything was moving. We've been working on hands for a long time and it just seemed like a good fit."

He said the biggest challenge for the project so far was that the premise is a very complicated computer vision problem; it's very technical. The scan not only needs to track your hands but understand what each of your fingers is doing. Hands can move quickly, hands can flip around, fingers can occlude one another. The team are making headway, but they want to deliver something "rock solid" in the next six or seven months.

"This is not a consumer product right now and it's probably a few iterations away from that," says Wang of the Nimble Sense's progress.

"It's only through developers and trying to figure out what the right SDK is for them - what the specs for the camera will need to be to support the interactions they want - that we'll get there. This is not the ultimate solution yet, it's a part of it. The other parts we're still trying to figure out."