Niantic is testing Pokémon Go on HoloLens 2 to let you play with friends remotely

CEO John Hanke talks about the leap to AR glasses, and figuring out telepresence

Someday, maybe, you’ll be playing Pokémon Go in a park, and a friend will find you and ask to join. Except they’ll be somewhere else, and they’ll join you as a hologram. You’ll play side by side. Will they be in your space, or will you be in their space? How exactly will that work?

Pokémon Go defined the idea of real-world mobile location-based gaming back in 2016, and became identified as augmented reality’s “killer app” because of its global popularity. Its developer, Niantic, has been extending into more immersive experiments: a partnership with immersive theater company Punchdrunk, and a collaboration with Qualcomm on concepts for future smart glasses. Niantic also announced it’s an early partner with Microsoft’s new collaborative AR/VR telepresence technology, Microsoft Mesh, and will be using the HoloLens 2 to test out playing the game in AR between people wearing headsets.

AR headsets are different than phones, both in terms of the tech and the metaphysics. Niantic founder and CEO John Hanke, in a conversation over Zoom, admits that the exploration is an experiment right now.

“The experiences that we build, we anticipate a world where there are multiple headsets in the market, there are going to be multiple platforms. And we want to get our experiences out broadly,” Hanke says of the upcoming future of AR and mixed reality. “I don’t think we’re at the point where there’s one ecosystem to rule them all.”

Hanke sees the HoloLens 2 as one of the best ways to test out experiences that may eventually run on other smart glasses. “It’s a great opportunity for us to get in really early and work on this stuff.”

Niantic already has AR and world-mapped shared experiences on its phone apps, but Hanke acknowledges that doing the same thing on a headset involves a different type of perception and interaction. “What Microsoft has not done much of with HoloLens, owing to limitations in the hardware itself, is outdoor stuff,” Hanke says, and Niantic is testing the HoloLens 2 by pairing it with phones to get mobile connection and GPS and using them in places similar to where phones are used. “Holding up your phone and seeing through the screen is one thing, but when you have the headset on and the Pokemon are all around you, and they’re 12 feet tall, it’s just magical.”

Hanke’s on-stage Microsoft Ignite appearance via VR showed Microsoft’s mixed-reality head Alex Kipman and Hanke appearing together, Hanke beaming in from a park in San Francisco. Kipman beams over to Hanke, and then a concept of what Pokemon Go in AR could look like is briefly shown.

Hanke says the goal is to work out key concepts in how people would interact: “What happens when somebody reports in from somewhere else? And what do they see? What do you share? It’s very much a tool for us to experiment with anticipating future hardware that would be, you know, more suitable for outdoor consumer use.”

AR smart glasses seem like they’re being developed from multiple corners of the tech world, from Facebook, to Qualcomm, to possibly Apple, Samsung and Google (and, maybe, at some point, Microsoft). But the social dynamics of AR glasses haven’t been figured out yet. Privacy issues haven’t been overcome. Those considerations, in a game or elsewhere, will need to be resolved.

Microsoft’s vision of blended realities and telepresence carries over from AR to VR and even flat-screen phones and PCs, but Niantic is testing its gameplay only on the HoloLens. “Our focus was peer-to-peer on HoloLens,” Hanke says. “It’s easier from a conceptual point of view, because everyone has the same set of assumptions and interactions. But theoretically, you could imagine a world where it spans phones and AR and potentially even VR, but we have not crossed that Rubicon yet.”

Hanke sees the idea of playing with a holographic friend as natural, because in a sense we’re with virtual companions all the time — via audio. “In our neighborhood, everyone’s talking on the phone, or listening to a podcast. This idea of having companionship while you’re out in the world is very, very natural. Taking that to the next level, having that be a sort of holographic companionship while you’re out … in a sense, it’s novel and sci-fi, but you know, in other ways, we’re kind of already doing it.”

Niantic’s partnership with immersive theater company Punchdrunk points to more theatrical, social experiences that could be on the horizon, too.

“That area of live entertainment, thinking about exploding that out into a mixed reality world where you could have people physically present and at a performance, you could map the real show to your house and have real performances happening with holographic players,” he says. “People are really interested in trying to figure it out. And I think it’s super exciting. Things have been kind of stale in the tech world for a while. This is like a whole new set of toys, particularly when people and co-presence are involved, it just sort of immediately gets exponentially more interesting.”

My Opinion

The AR Gadget could possibly be a massive hit for Pokémon GO Players who are interested in immersion. While using phone itself is already immersive enough, especially with the AR Camera on, some mobile phones may not be able to handle the AR Camera feature, and on the other hand, if players can afford a medium-high end mobile phones, they definitely can afford this Gadget. Since this is still a planned project, with a lot of the dimensions left unknown, I think this project builds up a great amount of interest and anticipation and overtime, their updates will be well awaited by Pokemon GO players worldwide.

Another big factor of this AR Gadget is that, since the gadget are able to connect to mobile devices in order to have GPS and internet connection, that means the gameplay players are going to receive are no different than the kind of gameplay that they’re used to using their mobile phones, of course, the main difference would be the immersion to the game itself. The fact that internet connection and GPS are able to be integrated with the device means that the gameplay could possibly be real-time.

The Conclusion

The Pokémon HoloLens 2 is a project developed by Niantic, with the help of Punchdrunk and a collaboration with Qualcomm on a concept for future smart glasses. Niantic will also partner with Microsoft to use their Microsoft Mesh and will be using the HoloLens 2 to test out Playing the game in AR between players wearing headsets. Since headsets are a completely different tech and metaphysics than phones, the headsets are still in experiment right now.

It is said by CEO John Hanke that the exploration are experimental. “The experiences that we build, we anticipate a world where there are multiple headsets in the market, there are going to be multiple platforms. And we want to get our experiences out broadly,” quoted from CEO John Hanke. He believes that they are at a point where only one ecosystem is required. CEO John Hanke believes that the HoloLens 2 is one of the best ways to test out experiences that can also affect the other smart glasses.

Niantic already has an AR world-mapped shared experience on phone apps, with the experience, combined with the hardware limited HoloLens, Hanke claims to be able to create a more immersive and “magical” experience. It is shown that Alex Kipman, and Hanke meeting while Alex is a hologram character next to Hanke in person.

Microsoft vision of blended realities and telepresence carries over from AR to VR and to flat screen phones and PCs, but Niantic is testing gameplay exclusively on the HoloLens. This HoloLens 2 are intended not only for theatrical experience but also social experiences too.

Referensi:

https://www.cnet.com/news/niantic-is-testing-pokemon-go-on-hololens-2-to-let-play-with-friends-remotely/