Telegram store your message and conversation history on their servers, whereas Whatsapp run somewhat like SMS - they pass your message on to the destination and then forget about it.
Clearly every point in the chain matters, but it always surprised me that a company that puts so much emphasis on security and privacy thinks it's a good idea to keep your messages in the cloud:
"Unlike WhatsApp, Telegram is cloud-based and heavily encrypted. As a result, you can access your messages from several devices at once"[1]
Actually, you have to assume that any encryption is a temporary measure; it can and will be broken in the future. With that in mind, if something is worth encrypting, it's possible you don't want the service to store it after it's useful.
Of course, whatsapp could register devices which are thrown away after x days without delivery then store the messages until delivered to all end devices but that's likely a core infrastructure update which would require them to store more data, not the same as just writing another client.
Still, what they've created as a web client is a little batty; typically I'd want to use those when I can't find my phone.
Clearly every point in the chain matters, but it always surprised me that a company that puts so much emphasis on security and privacy thinks it's a good idea to keep your messages in the cloud:
"Unlike WhatsApp, Telegram is cloud-based and heavily encrypted. As a result, you can access your messages from several devices at once"[1]
[1] https://telegram.org/faq