It has a long a storied history. The concept of running an app remotely and use a local display server is probably the thing that sets X11 apart from the rest of the pack. This is still really useful in my humble opinion -- I can run X11 apps (e.g. xv) on a headless EC2 server running Linux in the cloud and have the UI/display on my Mac (using Xquartz).
This story runs in parallel with the history of Unix itself.
The fact that there is an implementation of X on Mac speaks to the power of this system, IMO.
However, regarding remote setups, is X forwarding significantly superior to using something like VNC? I've never done a side-by-side myself, but most of what I've read indicates that VNC is usually faster.
As a semi-experiment, a few months ago I decided to try running Firefox in a Linux VM, and render it in XQuartz on my Mac host. The experience was not pleasant, to say the least. And that's what I'd think to be the best-case-scenario for latency. Hard for me to imagine using it over WAN.
There was a lot of work done to make X11 work better over low bandwidth/high latency links. [0] But in the end you could get most of the benefits just by forwarding your X session over SSH with compression enabled. Although X11 is old enough that compression would have used too much CPU back when it was developed.
The more interesting comparison between X forwarding and VNC is that X is really only displaying a remote application locally, not an entire screen or desktop. (Unless you use xnest to run X inside X which is pretty cool.) I used to have a FreeBSD workstation with a browser window running on a Linux server and a bunch of terminal windows spun up on different Sun servers, and you couldn't tell the difference between your local and remote windows. Remote windows are first class citizens in X11.
A better comparison with X is the RDP protocol, which too is an API level protocol (its basically forwarding large parts of the windows GDI interface).
OTOH, the big strength of RDP is the async stream requests. This gives it a much lower apparent latency than X, but it can also do the VNC thing and render/compress parts of the UI on the remote machine.
Depends on the era of apps. If you're doing mostly bitmaps (that is, Qt4+, GTK2+, anything that draws its own UI, anything with HTML incl all electron, anything using GL, anything using composition) then VNC will work better, as it transfers better bitmaps. Working with Xlib, GTK1, or Qt3-era or earlier apps, X will perform astoundingly better. Test with xemacs next time, and you should see your results inverted.
It has a long a storied history. The concept of running an app remotely and use a local display server is probably the thing that sets X11 apart from the rest of the pack. This is still really useful in my humble opinion -- I can run X11 apps (e.g. xv) on a headless EC2 server running Linux in the cloud and have the UI/display on my Mac (using Xquartz).
This story runs in parallel with the history of Unix itself.