Downvoting should cost something. Half a point, say. Or you only get a limited number of downvotes per week. Something.

Also, you should be required to give an actual reason other than "I disagree".

I know that's not what "the rules" say. "The rules" are wrong.