Ya I think v3 or v4 of my project will eventually go more into this realm and recreating things at a deeper level. Then maybe it could really escape the browser one day.

Would you mind sharing how you abstract the applications? I looked in your repo but I'm not fluent in React.

I ask because I have a very barebones Application concept in my WebXR environment and am curious what sort of issues you ran into supporting many different types of apps.

My current system is kind of like a Entity-Component system, once you start considering 3D elements and dependency resolution. My hope is to one day be able to allow users to upload script bundles as apps into their environment, with the environment being just a minimum viable VR desktop metaphor.

At this point, I only have a teleconferencing system and an "annotated Google StreetView exploration app" as Applications. I recently added support for YouTube video streaming that I did not do in my App abstraction, but now think should probably be done that way.