Augmented Reality in Mobile and Gaming

By Published on .

It seems like digital ad folk are already starting to get bored with augmented reality. But, thanks to the new iPhone 3GS, and hardware advances like Xbox's Project Natal, a gaming console add-on that uses video cameras as input devices, it looks like there soon will be an entirely new generation of AR to play with. (For now, this process largely utilizes computer webcams and black-and-white symbols on paper as means to interact with 3D graphics--see our Need to Know tech primer for background.)

Since the Super Bowl, when GE and Goodby, Silverstein & Partners wowed ad and tech worlds alike with Step into the Smart Grid, the first high-profile ad campaign utilizing augmented reality, the internet has been somewhat flooded with AR campaigns for marketers ranging from BMW to musician Julian Perretta. Since Smart Grid, these digital campaigns have been launching to skepticism among digital creatives: Why is there so much excitement around ad executions that look more like branded demonstrations of the technology than useful consumer tools? What actually is the potential of this technology? Does AR have a future once the initial appeal, one-off tech thrill wears off?

On his blog crackunit, Iain Tate, partner of digital agency Poke, recently raised these very questions, pointing to AKQA's U.S. Postal Service Virtual Box Simulator (above), which some have called the first execution that takes AR beyond tech toy to real utility for consumers. If this is utility, is AR the kind of technology people will eventually get bored of and discard?

"There are definitely amazing things that can be done with AR, I'm just not sure that the current crop of examples with fixed webcams are hitting the spot," Tate says in response to his post. "I think it's really about how it gets ported over to mobile devices in an interesting way."

The iPhone 3GS hit stores on Friday with a video camera, GPS and embedded compass--these new functionalities add up to a smart phone that can support AR. With AR applications on phones, users could potentially interact with graphics, 3D or otherwise, in the real world. With location-tagged information discerned by the GPS, or AR applications that can recognize real objects instead of only the black-and-white markers that activate most AR applications now, a mobile camera could be a portal to information physically linked to real world places and objects.

Bruno Uzzan, CEO of Total Immersion, a developer pioneering AR technology, says this could mean using mobile to access meta-information on places and things by simply viewing them through a phone's camera. For travel, Uzzan imagines viewing the Eiffel Tower through a mobile camera to see informative videos and links super-imposed over the monument. Or, for assembling furniture, instructional diagrams could be overlayed on parts of, say, an IKEA bookcase to bring the 2D drawings off a sheet of paper onto actual objects.

Uzzan, whose company developed the Topps 3D baseball cards, points to recent hardware developments beyond the iPhone that will enable a more utilitarian future for AR.

"We're seeing today some hardware convergency that will benefit AR," Uzzan says. "The new iPhone has video capacities, the Samsung phone, the DS console, it means that as soon as you have video function, you are ready to go to get access to AR and this will be easier than just using a PC. I really believe the next generation of gaming will use some AR components. And iPhone applications will use AR components. People say AR is a gimmick, because they are stuck on what's available today. This is an evolving technology. There's a ton of application that will happen in the next 12-24 months. We are in the time frame, the next iPhone generation is a very powerful device for us."


As for other smart phones, Amsterdam-based SPRXmobile launched Layar (above), which claims to be the first mobile AR browser, for Google's mobile operating system Android last week. Similarly requiring handsets with GPS, a compass and video camera, Layar identifies your location with the GPS, knows in which direction you're pointing the camera with the compass, and overlays information on the objects and places on the phone's screen. With Layar, your phone becomes a viewer that affixes images, text and video to real places. The application launches in the Netherlands with partners like ING for ATM information, Funda for real estate, and for healthcare, which supply the location-linked content.

Another mobile AR app currently in development for both the iPhone and Android lets users tag their own information to places and things. With Japan-based app developer Tonchidot's Sekai Camera (above), users can leave comments on the world, potentially transforming environments into giant, 3D social networks.


For gaming, Total Immersion's Uzzan also says that Xbox's Project Natal (above) introduced at E3 in Los Angeles this month, signals a future for AR in console gaming. Project Natal is controller-free gaming that allows players to interact by simply moving their bodies and speaking. Natal uses cameras to track the movements of players' skeletons and distance from the console. Where the mobile applications of AR use positioning systems to bring up location-based information, Natal technology could be applied to AR for motion- or human body-activated graphics. "We're starting to see large companies thinking about AR and using motion capture control," Uzzan says. "Project Natal is a good component for AR, the way you can start using your body to play a game--that could be combined with 3D and video. This is just the birth of AR, there's still a huge market in front of us."

Just as Project Natal used the body as input, there's also a future in "marker-less" AR--right now, many of the AR applications require users to print out black-and-white markers, which pop out into the 3D graphic when in front of a webcam. Uzzan says right now the technology exists to activate AR with real-life objects, like a Coke can or even a human face. Recently, for the release of the Transformers sequel, an AR app recognizes faces as the market and overlays a 3D robot mask (below).

Beyond hardware advances and marker-less AR, chief creative at alternate reality game developer 42 Entertainment Alex Lieu points to Georgia Tech's Zombie game as an example of how augmented reality provides an entirely new way to game using real-world components.

"Augmented reality provides a new way of interaction, like how Wii is a different way of interacting with your console," Lieu says. "It's not that we haven't seen earlier versions of this. Remember the virtual reality helmets that people could wear? It's just that the graphics were never that good, the experiences were pretty clunky, the technology wasn't really there to render it well, it was pretty geeky. But now we see more consumer-friendly applications that look great, performs well and people see how interacting with it is different from clicking buttons in a browser, or using a traditional console joystick."

So maybe all the demo-like ad executions of AR aren't so bad after all. While digital marketing's current obsession with AR runs the risk of over-saturation, which could exhaust consumers, on the upside, today's apps, however useless they may be, acclimate the general public to the technology and a future of 3D interaction.

"If you look at 3D and virtual world theory and how people have been developing user interfaces and those environments, whether it's everyday things or it's process-intensive things or gaming, a lot of people think 3D is the future of interaction," Lieu says. "This is one step closer for consumers to wrap their heads around what that might look like. This marketing front of AR is opening doors for industries like gaming to take this technology further."
Most Popular