As an aside, I always found it strange that eating wild land animals is looked down upon as "bushmeat" and unsustainable and unhygienic, while eating wild sea creatures is just "seafood" (with farmed aquafauna looked down upon as the inferior product).
That's a good point. There is plenty of seafood that when not dealt with properly will happily kill you or make you at a minimum severely ill. Mussels, shellfish, oysters are best avoided unless you are very sure about where they came from and how they were prepared.
Besides that, due to the liquid environment anything in the sea that is poisonous is usually so in a very potent fashion. Species jumping viruses are rare when it comes to regular fish but marine mammals are a risk.