Meta just revealed the first Quest 3 mixed reality gameplay, showing the passthrough quality and room-aware features.
Mark Zuckerberg posted the footage to his Instagram page, showing him playing a series of mixed reality games and demos on Meta's just-announced $500 headset.
Quest 3 will feature stereo color cameras for the first time in a Meta headset. The passthrough quality seen in Zuckerberg's video looks noticeably superior to Quest Pro's, lacking the "warping" distortion and graininess seen in Meta's now-$1000 headset.
One portion of the video appears to show environment relighting, a feature already present in mobile AR platforms like Apple's ARKit, Google's ARCore, and Snapchat's Lenses. This involves changing the colors and lightness of the real portions of the scene to match a virtual light source, in this case a window to a virtual aquarium anchored to the wall. It's an example of a feature passthrough headsets can pull off convincingly because of their full control of pixels while transparent AR glasses can't because of their limited field of view.
Another part of the video shows how these wall-anchored portals to virtual worlds can be used to extend the environment for mixed reality gaming, with a zombie enemy climbing through a virtual window to the room.
Yet another part shows how miniature games can be anchored to your desk for other surfaces for seated mixed reality content.
In the final section of the video Zuckerberg joins Meta's CTO Andrew Bosworth to play two colocated mixed reality games, meaning they see the same virtual objects at the same locations in the room, including ones they're holding. This can be achieved today with the Shared Spatial Anchors API Meta released in December.
The first colocation game shown sees Zuckerberg and Bosworth compete in a sword fighting game standing facing each other, while the other shows a more relaxed scenario of sitting beside each other remotely controlling virtual miniature tanks on the floor.
Meta still hasn't gone into detail about how exactly Quest 3's environmental awareness works. For mixed reality to interact with your walls today on Quest 2 and Quest Pro you have to arduously manually mark them out, but Meta confirmed Quest 3 has a depth sensor and claims it's capable of "intelligently understanding and responding to objects in your physical space."
Full details of Quest 3's mixed reality technology will be revealed at Connect 2023 on September 27.
