Cartoon Campout
This was an experimental project targeting the Meta Quest 3. I wanted to explore the usage of seamless portals to travel between video see-through and a virtual world.
Source Code
Introduction
In this very short project, the player can go through a seamless portal into a cozy virtual campsite where they can cook and eat marshmallows and watch old cartoons on the TV.
Technology and Context
The portal is created using a custom stencil shader that either renders a passthrough material (provided by MetaXR) or the virtual world. Although the player can never see it, the portal is actually a 3 dimensional mesh with inverted normals which ensures a seamless transition without clipping. Every iteration uses hand tracking only, no controllers are needed. The player can open and close the portal by pressing a physics button, cook and eat marshmallows by the campfire, and watch an old cartoon on the TV after turning it on.
Video Demo
This video shows the resulting experience running on a Meta Quest 3.
What I learned
This project was a short prototype I made to get used to Meta's XR SDK and the Quest 3's video see-through capabilities, before formally beginning development of my master's thesis. I had worked with XR before, in my curricular internship using the Hololens 2, but this was my first time working with a proper VR headset and video see-through.
I learned a lot about optimizing for mobile VR. The Quest 3 is a very capable device, but under the hood, it is still technically a mobile android device, so performance is fragile and it is very important to keep XR projects running at a steady framerate. I used the Unity Profiler extensively to identify bottlenecks and optimize the experience, and I also had to extra careful with polygon counts when using third party art assets.
Overall, I had a lot of fun making this small experience, and the portal effect seen here, that I had done mostly for fun, ended up being one of the core contributions of my master's thesis, after very extensive iteration and refinement.