a bunch of accounts
I believe you meant: fake accounts, not bunch of accounts ;-)
Itch does remove those, but it can take very long. And for some ratings, they might disagree. Or just forgot in all the swamp of other issues they get requests about.
I agree that people should only rate games they are interested in. They need not necessarily have played them, but it's just lame to rate things for which you are not the target audience.
For those situations with few votes you should also not underestimate counter votes. People that might not have rated at all normally, but see the game having a bad average and deciding to give it a bump, because they think the game is (literally) underrated. It's also a reason why this downvoting business will might achive the opposite what people think it would. A situation with a bunch of most likely fake accounts is completly different though.