I think many of the ills of this type this can be traced, at least in part, to TeX's only-partial whitespace invariance.
The shifts in behavior with an added newline (or absence thereof) or a comment, as above, simply wouldn't happen if the language ignored whitespace outright.
Of course, requiring tags for newlines/paragraphs would make raw TeX as un-readable as raw HTML. It might be worth the trade for the predictability, but for the era in which TeX was written, one without IDEs or LyX, I suspect that Knuth made the right choice.
The shifts in behavior with an added newline (or absence thereof) or a comment, as above, simply wouldn't happen if the language ignored whitespace outright.
Of course, requiring tags for newlines/paragraphs would make raw TeX as un-readable as raw HTML. It might be worth the trade for the predictability, but for the era in which TeX was written, one without IDEs or LyX, I suspect that Knuth made the right choice.