John Hanke, CEO of Niantic, held a “vision talk” at GDC 2019 , titled “The State & Future of AR Games: Rose-Colored Glasses”. Although we strongly recommend that you watch the video in full length (56 minutes), we are providing a written summary for our busy readers.
Here’s a link to the video because GDC videos can’t be embedded \_(o.o)_/
As a child, Hanke was fascinated with pull-out maps from National Geographic magazines. Today, he wants to augment the world map with AR and the road that leads there is called Niantic (must read: Secrets of Niantic Real World Platform).
Niantic popularized AR through Ingress and Pokemon GO, but the the real work is happening in the background, through continued development of Niantic’s Real World Platform. The Real World platform is the key component of Niantic’s AR future where people don’t use phones, but rather glasses to interact with AR. Surprisingly, the Real World Platform team is currently the same size as the Pokemon GO team. The end goal of having such a large team is to productivize the technology and to help it mature and support a shared, single instanced, AR experience.
Advanced AR and accessible games
A key part of the Real World Platform is Advanced AR, which Hanke described as a “common layer” between Apple’s ARKit and Google’s ARCore technologies.
Advanced AR is Niantic’s term for a set of technologies that make AR work outside platform boundaries. Niantic is offering a metric ton of money (“tens of millions” according to Hanke) in hopes that their Beyond Reality contest will find AR projects that can help sell the RWP as a product.
It is possible to build games similar to Pokemon GO, based on AR, which Hanke calls “accessible games”. According to Hanke and third party metrics, accessible games target completely different demographics than typical games:
- 41% female players
- >30% user retention
- Daily play
- Long-term retention
- Healthy spend
Hanke also warned that AR comes with a set of challenges that must be taken into consideration when designing AR games: social stigma from holding up your phone, phone ergonomics and real world obstacles.
Connecting Atoms + Bits
“I think it is one of those platform shifts you see once every 10 to 20 years. I’ve lived through a couple: we saw PCs, we saw the cloud, we saw mobile. I think AR is one of those shifts”
According to Hanke, the current AR is little more than a parlor trick. AR v2 is going to connecting AR objects (bits) with the actual location where they exist (atoms of real world). There are three core pillars to AR v2:
Mapping reality – a map for machines, robots and software. Think ray tracing but not in games, ray tracing the world around you. One of the problems Niantic faces when mapping the world is that some things simply change too often – people walking, cars moving, etc. Currently, mapping is limited only to permanent structures – roads, buildings, floors. Everything else is calculated in real time with machine learning and neural networks. For example, this is how Niantic’s software sees people and other moving objects:
Understanding reality – augmentation depends on understanding reality and data that describes it. You can’t augment a road if you don’t know it’s width, material, sidewalk height. The goal of Understanding is to use machine learning and similar algorithms to present the real world as data for AR software.
Sharing reality – multiplayer in AR requires super low latency, dynamic tracking and cross platform support. 5G is making big strides in enabling this and Niantic is already working with several companies — Deutsche Telekom, SK Telecom, and Samsung — to see how their respective 5G networks can benefit its products.
Hanke referenced Keiichi Matsuda’s Hyper Reality video as an example of a future where we could end up:
Matsuda’s video is not a positive vision and Hanke warned the audience that this is not the AR experience we should be building. In his mind, AR should augment the world around us and help us better connect with it, rather than forcing us to install Adblock on our AR enabled sunglasses. What a lovely, rose tinted, vision that is.