Now what goes with any kind of rating system is that it's always unfair no matter how you make it. All a downvote tells is that somebody dislikes something... and all an upvote tells is that somebody likes it. Big deal! If somebody tells me they don't like my game because it's too hard or too easy,then I know that I should alter the dificulty, and that is helpful. Problem is that most people cannot be bothered to go into details like those. However in the end upvotes are basically just as useless as downvotes. In fact, the only fair rating system is no rating system at all.
Nobody likes to be downvoted, whoever yelling about it, and writing in all caps won't change anything. If you feel something's fishy about the downvotes, then we got a problem. I remember on another site that somebody downvoted all my games there in less than a minute, and then it's safe to assume we are dealing with a clown who likes to downvote everything that comes in their path. I don't know how itch.io handles those kinds of clowns, but I only hope they'll ban them and remove all their votes, and that such a thing is reported is all natural.
I do know there are several sites who removed downvoting features in order to prevent flame wars. Some sites merely hid downvotes for the same reasons. Basically, when it comes to removing or hiding downvotes, I think when it comes to preventing flame wars. But one thing should remain clear. The possibility to downvote does exist for a reason, and there are people out there who downvote for the just reasons.