Email is tremendously useful, and whatever replaces it should borrow from those useful bits. It also has numerous problems. Some are listed in the article:
1. Clients (and some servers) mangle email. You can't be assured that what you send is what someone else will receive. I blame the clients and servers here, but ...
2. Privacy. This has become among the biggest downsides. Whether it's data-in-flight or data-at-rest, email remains an exceptionally poor choice for private communications, and there's been little real progress on addressing this, in large part because of vendor reluctance at both client and server / system level. E.g., Microsoft, Google, Yahoo, and AOL.
3. Users. Systems which work well with email rely strongly on compliance by users, which relies strongly on enforcement by communications managers or moderators. For LKML, that would be the kernel mailing list etiquette. Battles over header formats, quoting styles, and quote before vs. after exist because these strongly impact workflow and efficiency of groups as a whole. Failure to acculturise hampers the group, and acculturisation rates limit group size and growth.
Much of Linus's noted "personality problems" come from the task of having to corral a group of people over which he has little if any formal control, a decidedly limited-bandwidth social communications channel (e.g., no tone, expression, voice, facial gestures, etc. signals), and the necessity TO BE UNAMBIGUOUSLY CLEAR when someone is transgressing norms. (Yes, I'm aware of HN's norms on ALL CAPS.)
4. File formats. Both a blessing and a curse: you can send anything. This means any given group can specify its preferences, but also raises the problem that there are few global preferences given. Kernel development works well with email by having specific tools and formats, from coding style to patch and git tools which create workable code fragments, to conventions for list discussion. This is strongly linked to #3 above.
5. The directory. In thinking about how email compares with other proposed alternatives, one key is that email lacks the concept of a central directory. Or at least, not a formal one (spammers may trade in same). On Twitter, or Facebook, or HN, every individual user is uniquely identified within a global directory. In Email, by contrast, what exist are addresses, which is to say, <user>@<domain>, where the domain has its own sub-directory. But there's no guarantee that domain1 would know that user1 is some valid directory entry at domain2. At best, domain1 knows how to reach domain2, and attempt delivery, via DNS, generally though not necessarily through MX records. This becomes more of a problem with ...
6. Spam and spoofing. Because there's no global directory, the validity of an email from <user2>@<domain2> cannot be assessed. In fact, there's no requirement that that even be a valid address, though as a convention it's generally considerate and useful. There've been numerous hacks to add increasing levels of authentication to email (SPF, DKIM), but the protocols remain weak. There's a much broader problem of metadata leakage. Deriving from this:
7. Reputation. Much of the spam problem derives from poor tools and support for establishing, assessing, sharing, and acting on the reputation of specific senders -- whether individual originators, email hosts, domains, or IP ranges. Changes to how the Internet is being used, including very large aggregated email providers (e.g., Google, Microsoft, Yahoo, AOL), and point-of-origin masking systems such as Tor make formerly useful concepts of IP and domain-based reptuations of limited reliability. And a consequence is that the assurance of getting messages through to any one person are becoming quite low. The more users are added to a system, the higher the noise level.
Email was designed for a friendly, unencrypted, lightly used system of mostly known and fairly trusted users numbering in the thousands. It's scaled remarkably well, by roughly six orders of magnitude. But it's become exceptionally creaky.
I think you're trying to have your cake and eat it too. Wouldn't a global directory of email addresses be the antithesis of privacy?
The greatest difference between email and all of today's proprietary alternatives is that your email identity doesn't need to be vetted by any central authority. Of course this makes spam harder to deal with, but it's part of the cost of decentralization and one that we must learn to deal with. Fortunately, most of the email services I use are pretty good at filtering spam, as well as making sure that my emails don't get marked as spam by other email services.
My inclusion of directory as a downside of email was a poor choice on my part -- I occasionally get swept up in writing something, and the thought's been kicking around my head for a few days that email is NOT based on a global directory, but a directory of directories, possibly hierarchical. This means that if you want to get something somewhere, as with routing and DNS, you kick it off first in the right general direction -- someone who knows more about such things than you do, at a minimum.
So, you're right, the fact that email doesn't have a global directory is a feature. It's a feature that's underappreciated by a great many folks who've suggested alternatives to email that do rely on grand central unified directories.
Which suggests that an email alternative which has a similar directory-of-directories concept might stand a better chance.
I've been thinking along lines of clustered hierarchies of users, with various trust bases within them. If you wanted clusters of no more than about 50 neighbors (well within a Dunbar exhange), then a hierarchy 6 layers deep gets you to coverage of all of Earth's population (15.6 billion individual nodes). Deeper, cross-linked, or other structures could offer greater depth of address use for individuals.
I've also been wondering if perhaps national postal systems ought to take over electronic messaging. I can hear the screaming already, but with privacy-supporting protocols (including metadata), much of the risk might be mitigated. In particular, I'm interested in addressing the "who are you?" question -- identity, and re-establishing it, is difficult.
I'm thinking that a good reputation system, and perhaps a poll-and-fetch method, rather than a straight delivery, could make email much more resistant to spam.
Under poll-and-fetch, a sender would indicate that it has a message for another party. That party would then fetch the message, should it be interested. Fully vetted senders would have their content fetched automatically. Spam senders would have to sit on their spools and wait for (and support) many individual incoming fetch requests. Spammers identified as such would be identified through reputation systems.
Granularity of reputation might be at IP space, domain, mail host/peer, or individual sender basis.
For many sender/recipient pairs, a strong and clearly positive or negative relation would be established early. The hump is to become known as a trusted sender, for which some sort of trust seed or reputation vouching system might work. I'm still hazy on what might work there.
Your poll-and-fetch suggestion sounds really interesting. Even if is not combined with a widely used reputation system, it would greatly increase the cost for spammers by requiring them to stay online indefinitely. It's basically graylisting on steroids, but without the annoying delay that graylisting imposes on every sender.
Mind that that cost can be met by some spammers, as it's effectively how a number of present online systems work. But in general, spammers tend to be taken down, and you remove the fire, cram, and forget mechanism of mail blasters, or the sparse, wide botnet model, where a given node emits only a relatively small number of messages.
Adding reputation to the mix strikes me as the secret sauce though.
Thanks for that. Yes, djb has thought through a lot of matters of trust and ought be considered. I'll take a look at that, I think I've read it before.
Number one on your list is a big issue with using e-mail for kernel development. If you want to submit patches by e-mail, you have to install and configure one of a handful of clients that doesn't mangle the patches first. There's a list in the kernel documentation and they're not terribly user friendly. (In fact, by necessity they have to be, because not wrapping lines automatically is terribly user-unfriendly and required by kernel development. So the only clients that meet the criteria are the ones that don't care about being a pain in the ass to use, and this permeates the entire user experience.) This isn't an issue for the core kernel developers because they already have a suitable e-mail client configured and installed which they're used to using, and because they're generally not submitting patches that way anyway.
If you're sending email, it's trivial to use, if all else fails, a shell tool or application-based email client. Standard unix mail, mailx, mutt, or various mail utilities within Perl, Python, Ruby, etc., can all be adapted to accept patches and some cover text with ease, largely noninteractively.
If you're actually reading email off LKML, that's another story.
But seriously: mutt, pine, Emacs mail mode, or even graphical clients such as Evolution or Kmail, will work just fine. And can be used in conjunction with nontechnical (there's another phrase I'm avoiding using here) GUI mainstream email clients used for other tasks.
If you can't figure out how to install and use mail / mailx / mutt with git, you probably shouldn't be submitting kernel patches. Which in this case is a useful filter.
Wait, I thought patches are typically attached, not sent in-line on lkml? Admittedly I've only ever submitted a patch once, in 2002 or something - and at the time I was using pine as an email client.
People who are into kernel development are unlikely to be defenceless against garbage email clients. They're actually the type of people who can write a script in an afternoon to solve the issue.
> you have to install and configure one of a handful of clients that doesn't mangle the patches first.
git-am and git-send-email are used by some people. Submitting patches using those tools is just a matter of setting up the necessary parameters (email server and authentication information). But you can use any email client to participate in discussions about a patch series regardless of whether or not it mangles text (since review comments or further discussion obviously do not need to be applied to one's local copy of the code).
Regarding 5. what's wrong with opt-in and mandatory email lists?
If you want to reach a team, email team@example.com. When you join a team, email team-subscribe@example.org, or maybe your hr/sysadm will add you to team@example.com. If you want to reach person@example.com - if you're part of the same company, your client does an ldap lookup. Or you have a look at https://example.com/employees.html or contact.html.
It might at times be difficult to find a random person's email - if you know nothing other than the name - but it's generally very easy to find a contact email for a Dev/security team for a project or company?
I think the matter of lists is orthogonal to the issue of global directories.
I have ... well, not quite nothing, but little against the concept of email lists. They're useful. I've used them extensively. They don't, of and by themselves, address numerous other problems with email. And they can and do insert a few of their own.
Regards lists and directories, I'm not entirely sure what your argument is, but I can interpret your comment at least two ways:
First, an email list is a directory. Yes, it is. It is not, however, a global one. In the right contexts, a list can address the question of discoverability, but that's somewhat secondary. There's the question of what mail is or isn't allowed on-list (e.g., is it open to all senders, or subscribers only?). Service lists -- the effective utilisation of addresses such as postmaster@ or webmaster@ -- typically have to be fairly widely open. This creates some well-known issues, mostly having to do with spam, but also of privacy and organisational confusion over who should or shouldn't be receiving such messages, or how they ought respond. For many corporate distributions, the names of such lists are not generally advertised outside an organisation, though likely targets include "all@", "sales@", "engineering@", etc. Again, this can be useful or not.
The other point you seem to be making is that an organisation can automatically manage its own lists through directory systems. Again, this is addressing the concept of a directory of directories, and any one of the (sub)directories might choose to expose or conceal elements to specific entities. Nothing in any of this is required or constrained by what I've written above.
More generally, you seem to be addressing the question of discoverability, which is a general one. The idea that organisations with their own individual entities might have or conceal means of reaching or contacting people is an old one. I'd call it the introductions problem.
An anecdote from personal experience. At various times, I've had an interest in contacting specific people at organisations. Two come to mind -- an IBM director over a matter somewhat concerning the company, and an issue involving spam transiting a Microsoft-run service. My general biases are that I'm favourably inclined toward IBM, and unfavourably toward Microsoft.
In each case, I found a general switchboard phone number and called, asking for the individual by name.
In the case of Microsoft, the operator said "just a moment", connected me through, the director picked up on the first ring, we had a brief conversation where I described the problem (about five minutes or less), he said he'd get me in touch with the person I needed to talk to, took my number, and within fifteen minutes I was talking to the manager in charge of the unit. We worked on the issue (I had better information than him) for a few months whilst they resolved it, quite measurably.
In the case of IBM, I was connected to an administrative assistant who said, and I think I'm quoting directly, "You can't talk to him, he's a director".
I'm struck by the difference to this day. Much as I don't care for Microsoft, my impression is that they know how to run a business and to transmit information. On the other hand, IBM may have its reasons for sheltering its upper-level management. I suspect otherwise.
Interesting points. I guess I need some clarification as to what your perceived problem is - it sounds like email is equivalent to the phone system - are you saying you prefer facebook etc as a global directory, as opposed to specialized directories run by organizations?
The directory issue ties in with several of the other problems of email, and how you want to address them.
If we're assuming pervasive use of mobile devices, with no fixed static IP addresses, and quite possibly dynamic IPs even for stationary devices, several of the avenues to establishing reputation, identity, and routing get difficult.
If the idea is to offer a very large number, rather than a small number, of messaging hubs (equivalent to, though offering more services than, an email server), then issues of reputation, configuration, and administration come into play. You want something that's simple, bulletproof (not in the spam-friendly sense), robust, and useful.
Many erstwhile email replacements are premised on a single global directory. I don't think this can work, nor is it on the whole desireable, though it's certainly simpler to get started.
For point-to-point messaging, especially if we're including realtime protocols (chat and voice/video, not just text/files), you need a system in which either scheduled contacts can be made, or in which at least certain trusted/whitelisted parties can reach a given device, regardless of location, within seconds, or at the outside, a few minutes, so long as both parties are online.
Avoiding central peering and routing structures should be part of the system, for privacy and resilience reasons.
Addressing a few of the other issues I mentioned above, including standards for comms and the like, should also be supportable.
Your list of complaints is approximately my list of laurels, but many of your complaints are not specific to email.
1. Clients (and some servers) mangle email. This isn't a complaint about email; it's a complaint about bad software in general. It's also not my problem; I don't use such clients or servers and it's up people who use them to work around these flaws.
2. Privacy. Real privacy is achievable via email with PGP; everything else is no different than literally any other form of communication. Privacy will always be between Alice and Bob. The transport layer is just a transport layer. With email, those who want it know where to get it. Sure, it's not simple, but if it were simple it would not work. With other solutions, it's frequently not even almost an option. I don't get to audit Whatsapp's server logs; who knows what agencies have access.
3. Users. All systems of communication rely on compliance by users. Just about none of your thoughts on this topic are specific to email. There is no 'go fmt' for human expression, and whenever someone tries, human expression is no longer present. Always speak to your audience, even in an email. It's up to you to decide if a given audience is important enough to acculturate to their norms.
4. File formats. You can send anything -- thank god.
5. The directory. This has no place in the layer. It's like complaining that they didn't build the concept of a web sitemap into TCP/IP. Twitter, Facebook, and HN are one-to-many communications platforms, some of which have hacky ex-post-facto one-to-one mechanisms. Email has been n-to-n from day one. Pretending that it is universally desirable to map addresses to individuals and vice versa is incredibly naive. I not only have several email addresses for different roles, but I have several phone numbers and several postal addresses for different roles. On twitter, facebook, and HN, I would have to jump through hoops to achieve this, and on some services it's even against the TOS. This lack of outside force regarding how I handle email messages is its greatest strength -- I can trivially set up a mailing list, which is nigh impossible on, for instance, twitter, and utterly impossible on hacker news.
6. Spam and spoofing. Absolutely a problem. Will remain a problem until solutions get specific buy-in from multiple interested parties -- and that last part makes all the difference. Each time some SV megacorp institutes a real-name policy, or a captcha, or or or... another class of users is sidelined, disenfranchised, or otherwise excluded from the communications process. Email does not have that, because there are standards to adhere to, and some random project manager does not (quite yet) have the power to shove new standards down our throats. Thank God.
7. Reputation. This is now and will continue to be the universal problem across all of the internet. Solving this problem by trusting a third party service provider to do all of your vetting for you will fail, because all they do is push that problem downhill to the users. Joe Random wants to be friends on Facebook. Now I have to do casework to decide if I want Joe to access my inbox, and if Joe is even Joe. Susan User would like to add me to her professional network on LinkedIn. Same problem. The only difference is that when I run an email server I maintain agency to manage the entire scope of these decisions, instead of trusting that Facebook or LinkedIn is reasonably sure that Joe might actually be Joe, or that Susan is not in fact a ten-year-old Martian girl mining my profile for password recovery answers.
Is email perfect? No, of course not. But it and IRC are the closest things we have to level playing fields for someone willing to do the work to take their place as a first-class citizen of the internet. They will only ever be replaced by standards; no proprietary product will ever unseat them.
Mail clients written in the eighties can still work today. Mail stores created then are still readable by today's clients. The protocols and processes involved have definitely changed, but they've _evolved_, rather than getting thrown in the trash and replaced with version 2.0, then 3.0, etc, ad infinitum, ad nauseam.
Given the choice between an evolving collaboration and reliance upon the goodwill of some stranger I can't even name, my choice will always be the former.
I disagree with some of your points, but also want to note that what I gave as email's weaknesses are frequently also strengths. I'll try to clarify the distinction.
1. Clients: There are cases in which functions which ought be standardised aren't. These are particularly problematic where archives of email from multiple systems are presented. References, quoting, and reply styles are three long-standing friction points. Various conventions for noting changes in documents -- GUI client users relying on word colour won't play well with those using text-oriented platforms. Including now a great many mobile email clients, not just mutt and pine users.
Standardisation is what makes more complex interactions possible, whether you're talking messaging formats, file formats, or nuts and bolts sizing. This makes for an interesting though likely side conversation. My point is that the lack of agreement across major families of email clients makes more complex email interactions quite painful, something I've noted for decades.
The biggest problem I'm seeing is that client tool vendors simply aren't addressing numerous areas of concern, such as those I've listed here, or working in the least toward interoperability. This is for quite understandable reasons (gaming and abusing standards for selfish gain is an old trick), but it doesn't make the resulting situation any more desirable.
2. PGP/GPG doesn't address metadata privacy at all. It's not supported by many clients (see above), and has posed very long-standing issues by way of key management and web-of-trust issues. Moxie Marlinspike is among the voices suggesting PGP may have to be swapped for something else. I'm not sure I agree with his assessment, but his isn't a voice to be trifled with.
Without encryption at the transport layer, and on a guaranteed basis, privacy protections simply don't exist. The level of concentration of email services leads to other massive concerns. Much of that is due to a lack of sufficient trust mechanisms built in at the peering level, so one problem (spam + trust) feeds another (privacy).
3. The issues of user idiosyncracies is amplified by other factors, including the client issues above. Tools to standardise formats, or convert to/from various standards, might help. My primary issue is that problems in one area of the messaging world amplify those in others. It's a mess, and neither users nor tools are helping much.
I'm aware that people aren't robots. Not all of them, at least. Providing tools that make it harder to do the wrong thing would help considerably. That's not what we have now.
4. File formats: again, it's a matter of standards. Negotiation of what is/isn't permitted, and/or required, on a given conversation, for example, might go a long ways to addressing this.
5. The directory. Addressed elsewhere. More a characteristic / non-characteristic of email as a whole, though one many proposed replacements get on the wrong side of (trying to provide a total global directory).
6. Ties in to clients, security, authentication, privacy, and a mess of other factors. It's a problem, but it's a problem grounded in virtually every other part of the email system.
7. Reputation: see above. Though I agree that it's not something that can be addressed through a central agency. Another much longer discussion.
Again: the point in my list is to note where email fails most egregiously. I like email. I've used it for thirty years. I've defended it, strongly, against numerous previous criticisms. But I've come to change my mind, in that several of the fundamental premises and capabilities of it are now fatally flawed. It needs a massive refresh.
1. Clients (and some servers) mangle email. You can't be assured that what you send is what someone else will receive. I blame the clients and servers here, but ...
2. Privacy. This has become among the biggest downsides. Whether it's data-in-flight or data-at-rest, email remains an exceptionally poor choice for private communications, and there's been little real progress on addressing this, in large part because of vendor reluctance at both client and server / system level. E.g., Microsoft, Google, Yahoo, and AOL.
3. Users. Systems which work well with email rely strongly on compliance by users, which relies strongly on enforcement by communications managers or moderators. For LKML, that would be the kernel mailing list etiquette. Battles over header formats, quoting styles, and quote before vs. after exist because these strongly impact workflow and efficiency of groups as a whole. Failure to acculturise hampers the group, and acculturisation rates limit group size and growth.
Much of Linus's noted "personality problems" come from the task of having to corral a group of people over which he has little if any formal control, a decidedly limited-bandwidth social communications channel (e.g., no tone, expression, voice, facial gestures, etc. signals), and the necessity TO BE UNAMBIGUOUSLY CLEAR when someone is transgressing norms. (Yes, I'm aware of HN's norms on ALL CAPS.)
4. File formats. Both a blessing and a curse: you can send anything. This means any given group can specify its preferences, but also raises the problem that there are few global preferences given. Kernel development works well with email by having specific tools and formats, from coding style to patch and git tools which create workable code fragments, to conventions for list discussion. This is strongly linked to #3 above.
5. The directory. In thinking about how email compares with other proposed alternatives, one key is that email lacks the concept of a central directory. Or at least, not a formal one (spammers may trade in same). On Twitter, or Facebook, or HN, every individual user is uniquely identified within a global directory. In Email, by contrast, what exist are addresses, which is to say, <user>@<domain>, where the domain has its own sub-directory. But there's no guarantee that domain1 would know that user1 is some valid directory entry at domain2. At best, domain1 knows how to reach domain2, and attempt delivery, via DNS, generally though not necessarily through MX records. This becomes more of a problem with ...
6. Spam and spoofing. Because there's no global directory, the validity of an email from <user2>@<domain2> cannot be assessed. In fact, there's no requirement that that even be a valid address, though as a convention it's generally considerate and useful. There've been numerous hacks to add increasing levels of authentication to email (SPF, DKIM), but the protocols remain weak. There's a much broader problem of metadata leakage. Deriving from this:
7. Reputation. Much of the spam problem derives from poor tools and support for establishing, assessing, sharing, and acting on the reputation of specific senders -- whether individual originators, email hosts, domains, or IP ranges. Changes to how the Internet is being used, including very large aggregated email providers (e.g., Google, Microsoft, Yahoo, AOL), and point-of-origin masking systems such as Tor make formerly useful concepts of IP and domain-based reptuations of limited reliability. And a consequence is that the assurance of getting messages through to any one person are becoming quite low. The more users are added to a system, the higher the noise level.
Email was designed for a friendly, unencrypted, lightly used system of mostly known and fairly trusted users numbering in the thousands. It's scaled remarkably well, by roughly six orders of magnitude. But it's become exceptionally creaky.