Mozilla Hubs is the best 3d browser-based spatial audio platform I've tried yet by far. I hosted a virtual party in the first lockdown back in April and it was pretty fun and far far better than a zoom call due to the spacial audio. Unfortunately it did become a bit unstable past 15-18 people and you'd have to rejoin the session frequently when it started to crap out under higher load.

There already is a popular platform that fits what you are describing: VRChat.