Holodeck: Holy Grail or Hollow Promise? Part 1

For years game developers, game players, science fiction fans (generally) and Trekkers (specifically) have been told - and have been telling themselves - that the trajectory of video game history leads inevitably to a Star Trek-inspired Holodeck.

Since first bursting onto the scene in the 1980s, Jaron Lanier, generally credited as the first person to describe the potential of "virtual reality," has argued that something like the holodeck was unavoidable. Since then, he has been one of the leading evangelists for a fully immersive future. As recently as June of this year, in New Scientist magazine, he reiterated his desire to see that dream become a reality:

"I believe that the Holodeck as Holy Grail has the potential to lead us down a blind alley toward a dead end future"

Instead of thinking of it like a very 3D movie, or a video game that you're inside, I think what virtual reality is going to be like is a new kind of a medium where you're playing with your own identity, and that's what's so interesting about it… it's almost like you're exercising these forgotten little corners of your brain, some really old corners that evolved to actually control different bodies deep, deep, deep back in our evolutionary past. And that kind of very profound, intimate sense of experience is really what virtual reality's all about.

Similarly, interactive media thinkers like Janet Murray have written persuasively about the appeal of the "Star Trek future." In her 1997 book, Hamlet on the Holodeck, she discusses the role of immersive simulation in the future of narrative laying out an explanation of, and template for, a future rich in virtual reality experiences that is still relevant 15-plus years on:

"The format that most fully exploits the properties of digital environments is not the hyper-text or the fighting game but the simulation: the virtual world full of interrelated entities, a world we can enter, manipulate, and observe in process."

If you haven't read Hamlet on the Holodeck you should probably do so - it's kind of a must-read for any game developer. I'll warn you, there are some very smart games industry folks who find the book as frustrating as it is educational! Still, a book that engenders strong feelings and gets people arguing is worth a read, right?

More recently, at USC, there's a team working on something called "The Holodeck Project". (Clearly, though smart, talented, and driven by a Star Trek inspired mission to "go where no man has gone before" they need a marketing person to work on their naming skills). Regardless, here's what they have to say about the goals they've set for their work:

The holodeck has given us woodlands and ski slopes… figures that fight… and fictional characters with whom we can interact. Or so we were promised many years ago by a certain Jean-Luc Picard. But now we actually might have an actual Holodeck to actually run around in and actually fight baddies in… [T]he Holodeck is a bit different to the [Oculus] Rift: It's not just a head mounted display, it's a full virtual reality experience.

"I have a rule that's stood me in good stead over the years - never build a game that depends on potential buyers owning a peripheral in order to play"

And finally, if anyone requires more evidence of the pervasiveness and persuasiveness of a Holodeckian future, the most recent sighting I've seen can be found in an article by Jeff Grubb, writing for the VentureBeat website:

The Holy Grail of immersive gaming is Star Trek: The Next Generation's holodeck - a room that you can enter that becomes an interactive experience and overwhelms your every sense. It's a concept so far-fetched that it still feels like we're a hundred years away from it, and we probably are… For something to qualify as a holodeck, it must trick your every sense…

I can't go where he's going, any more than I can go all the way with Lanier, Murray, and other VR/Holodeck evangelists.

In fact, I believe that the Holodeck as Holy Grail has the potential to lead us down a blind alley toward a dead end future.

I realize I'm swimming against the current here - what with the slavering anticipation for the Oculus Rift VR rig, reports of Holodeck-like projects from Sony and recent Microsoft Kinect-driven VR patents. But hear me out.

Here's my contention:

If we're careful and thoughtful in our approach to predecessor technologies like virtual reality and augmented reality, as well as to Holodeck-style full immersion down the road, we might end up in a great place - a place as compelling as the world Lanier, Murray, Grubb et al envision.

But if we're not careful, if we don't consider what VR, AR and Holodecks can and cannot do well, we'll just end up spending a lot of money and expending a lot of effort giving users something we think is cool, something those users think they want, but which will inevitably disappoint.

"Folks who go first often go bust. Players who buy early can end up broke"

This may seem obvious - do smart stuff and good things happen - but from much of what I read and hear, from all the gushing over headsets and new peripherals for interacting with things on the screen, it's obvious to me that a lot of people aren't thinking about the pitfalls ahead. For that reason, I'd like to go through some potential problems we'd better think about - and soon.

Pitfall #1: Do people really want VR, AR or Holodeck enough to buy a peripheral to experience it?

I have a rule that's stood me in good stead over the years - never build a game that depends on potential buyers owning a peripheral in order to play.

No matter how cool the game or how cool the peripheral, only a portion of the potential audience will have (or be willing to buy) something new to get the full experience of your game.

Heck, even a peripheral that's packaged with hardware can be a sales-limiting factor. Sometimes the old ways are the best ways and, today, lots of people are just fine with a keyboard or controller - no surprise given that these ways of interacting with a computer or console have been refined to razor-sharp effectiveness by thirty-plus years of use and mastery. People like a controller connected to a TV or monitor they already own. Very few, if any, "normal humans" want or need more immersion than that. Who wants to learn a whole new way of experiencing and interacting with a virtual world? By and large, people just want to play. Anything that gets in the way of that reduces the likelihood of people playing.

Clearly, there are some big hurdles to overcome here. You can say we're still in early days of gaming, let alone VR, AR and Hdeck. You can say, of course prices are going to be high and sales limited to early adopters. In other words, you can say that an audience limited by the need for a peripheral is okay - we have to start somewhere, right?

Not so right, I fear. Folks who go first often go bust (see Cybermaxx and Forte). Players who buy early can end up broke (yeah, I'm looking at all you Apple Newton owners out there). It's all too likely that the lack of a sufficiently large audience will lead to lack of developer and publisher support, which will lead to peripheral creators running into a brick wall we in the trade call "no money". And no money for anyone on the creation-distribution-play spectrum, regardless of where they fall on it, means the Holodeck (and VR and AR) may be further away than current technology would lead one to believe. They may never catch on…

In my book, when you require a peripheral "purchased separately," as they say in the commercials, you always lose. Anyone see peripheral-free VR, HR or HDeck? Uh-huh. Didn't think so.

Pitfall #2: How do you control this crazy thing?

One of many things we learned in the VR gaming circles of 20 years ago was that the range of mobility of the human neck limited a player's ability to experience even the most meticulously designed and immersive virtual world. Unless you were willing to stand while you played, or you sat in a chair that swiveled 360 degrees, VR didn't add enough to the experience of play to balance out the cost and deficiencies of the hardware. (And watch out for deadly python-like cords as you rotate to take in your oh-so-compelling virtual surroundings).

"How many people really want to walk, run and jump to navigate a virtual world?"

And that is just the tip of the iceberg when it comes to interfacing with a virtual world. Falling into a 3D space… feeling like you're really there in an alternate world… In a very real sense, that's the easy illusion to deliver to users.

The problem of how you interact with an illusory immersive world? Now that is one tough problem.

Creating a good head-tracking headset, while cool, is just the first hurdle. Assuming the full immersion of a Holodeck is even possible, and it seems likely it will be, probably sooner than anyone expects, what about the interface between user and device?

In a VR world, are we supposed to use a Wii Remote or something like it? In an AR world, are we supposed to flap our arms around, looking rather foolish as we do so? In a Holodeck experience are we supposed to run, jump, kick and punch?

How do you interact with a virtual world? (Remember, your solution has to be as seamless and refined as the mouse, keyboard and/or controller…) Who among us wants to walk and run and actually swing a sword for hours? How will you ride a virtual horse, climb a wall, or pick up the inevitable crate?

Head-tracking sounds great. If you want to look up, just look up. Check out your flank and turn right or left? Totally do-able and cool. Look behind you?… Oh, wait… hm… maybe do a full Exorcist and swivel your head 180 degrees? Maybe you have to get up off the couch and turn around? Use a mouse or keyboard you can't actually see in VR - and don't want to see in AR or a Holodeck?

Put another way, how many people really - really - want to walk, run and jump to navigate a virtual world? How many people really want to swing a sword for 5-10 (or 100) hours? I think we can agree on the answer - not many, for all the enthusiasm expressed.

A TV screen, a mouse and/or a keyboard are looking a little better, aren't they? If the new VR/AR/Hdeck peripheral you have to go out and buy makes such things harder rather than easier, why bother? You might as well go join a gym and get some exercise that way.

It's tough but not impossible to imagine solutions to control/interaction problems in a VR world, but being the immersive tech that's closest to what we already have, that's not much of a surprise. AR is certainly going to be harder, UI-wise (and that's assuming you can solve all sorts of visual and occlusion problems!). In an AR environment, you start with the movement problems I already mentioned, but then you also have to deal with the problem of interacting with real objects and virtual ones (and having the virtual objects then interact themselves with the real ones!). AR kind of makes my head hurt… The Holodeck? I don't even want to think about the interface and interaction problems there.

And remember - if doing things in a VR/AR/Hdeck world is harder than it is with the tried and true of a controller refined over decades of use, what use is a Holodeck at all? An immersive experience that, thanks to clunky UI, constantly reminds you that you're in a virtual world is going to drag you in and out of the experience every few seconds. That's makes it extremely unlikely users will be able to get into the desired flow state than they can in older, seemingly less immersive virtual environments.

Check back tomorrow for part 2 of Spector's column on the Holodeck, in which he offers up the remaining pitfalls as well as a confession

Part 2 is now available - read it here