I think poor people in the US are against legalization mostly due to the decades of “war on drugs” propaganda or other forms of conservatism (eg religion), not because they’ve seen people close to them being destroyed by drug use
These same sources also mistake causality, as many folks with mental health issues self medicate, rather than having drugs be the absolute source for mental health issues. Example: Cleon Skousen.