HTTP won out because it's the 'universal' protocol: you GET a resource, you say 'I prefer text/html but really anything is fine', the server puts bytes on the wire, it includes some metadata (headers, Content-Type), and then your user-agent interprets that Content-Type and displays the result. It's an extensibility advocate's dream. Using this and the jack-of-all-trades datatype HTML, we developed documents that link to other documents. When we were no longer okay with static pages, we hooked up programs that wrote HTML onto stdout and at the end of the day, everything just came across as a sequence of bytes. There's no formalized, official application-level logic to the HTTP state machine (although there are third-party attempts [1]).
Using these universal building blocks, we built applications where state transitions consists of GETs and POSTs. Eventually, when we wanted machine-structured data, we did XML-RPC, later codified into SOAP, before the backlash against hard-to-understand standards led to JSON being traded between server backends and client-side obfuscated, minified Javascript state machines.
Not enough people make new running-on-TCP or running-on-UDP protocols because new protocols are hard to design, they don't work with the one application where everyone spends 70+% of their time (the web browser), and they probably get blocked on a middlebox except if you use port 80 or 443 and fake being HTTP anyway. For all but very specialized use-cases, vomiting blobs of JSON (or if you want to feel extra good, some custom binary serialization format like protobuf or Thrift or Cap'nProto or MessagePack) across HTTP endpoints is pretty okay.
HTTP "won" for the reason that port 80 was open anyway on corporate firewalls, so everything was shoehorned into it - but not for purely technical reasons.
Oh absolutely [1]. But the HTTP/2 endgame is likely to re-define it in terms of a protocol over QUIC, a situation the QUIC folks are eagerly anticipating [2]. This is no surprise considering both originated at Google.
QUIC is a secure transport protocol (subsuming most of the features of TCP and TLS) that runs on top of UDP (because they wanted to craft a 'better' TCP, and the only other not-blocked-by-default transport protocol is UDP).
Since Google pushed it into Android Chrome, I cannot log-in into our customers WLAN infrastructure on Android 4.3 only with 4.4+ devices, not even by disabling it via the flags menu.
And some of our devices are on 4.3 and now need to make use of HSDA for network access.
Using these universal building blocks, we built applications where state transitions consists of GETs and POSTs. Eventually, when we wanted machine-structured data, we did XML-RPC, later codified into SOAP, before the backlash against hard-to-understand standards led to JSON being traded between server backends and client-side obfuscated, minified Javascript state machines.
Not enough people make new running-on-TCP or running-on-UDP protocols because new protocols are hard to design, they don't work with the one application where everyone spends 70+% of their time (the web browser), and they probably get blocked on a middlebox except if you use port 80 or 443 and fake being HTTP anyway. For all but very specialized use-cases, vomiting blobs of JSON (or if you want to feel extra good, some custom binary serialization format like protobuf or Thrift or Cap'nProto or MessagePack) across HTTP endpoints is pretty okay.
[1] https://github.com/for-GET/http-decision-diagram