This is exactly what I did. Also of interest is that Yubikey OTP is directly supported, you can host static websites (I generate my site with Jekyll and then rsync it to fastmail's web server), and you can have a ton of other aliases for your mailbox (I've got like 12 addresses right now that all point to fastmail).