The disappearing statements / transaction history is especially annoying.

A few years ago I had to review transactions for tax purposes, but I couldn't because they only let me go back 12 months. Even now my bank only goes back 3 years, while Amazon still has my order history from 1999.