> I don't think people will start pointing at things in the air.
There are other possible input devices. For XR glasses, the most natural would be some sort of gaze tracking. Another option would be an indirect device like a mouse or trackpad (most desktop and laptop users don't point directly at things in their screens), or even a set of cursor keys for menu selection.