Right now, in ranked game jams, games receive a penalty to their score if they receive fewer than the median number of ratings. I wrote a whole thread last week about why I dislike this system. However, I have since found the response by leafo (an admin) to another user asking about the scoring system last year, and realized my original thread was really more of a suggestion phrased in the form of a question. So, let me rephrase my thoughts in a more appropriate forum:

I understand the problem that the score penalty is trying to solve, and agree that it is a valid problem- if a game has a tiny number of ratings, its score is likely to be very inaccurate. However, using a score penalty to reflect this is highly misleading to devs: it tells them that their game was worse because it was less popular. As leafo notes, players usually ignore a game when they think the game doesn't look good, so there is at least some correlation between popularity and quality. However, this isn't always the case - I've played great games with terrible thumbnails and bad games with beautiful thumbnails. Plus, a game looking bad isn't the only reason a player might choose to ignore a game. For instance, I am far less likely to click on a game whose thumbnail makes it look like a horror game- even if it looks like a well-made horror game- because I don't like horror in general.

My suggestion is to replace the score penalty with a system that simply leaves a game out of the rankings entirely if it does not receive enough ratings- or at least allow game jam hosts to switch between the two systems/turn them off entirely. Leaving ignored games unranked has been proven to work well - Ludum Dare has been using this system for years - and it still keeps the benefits of the score penalty: encouraging community interaction (so devs can get their games ranked) and preventing largely ignored games with skewed ratings taking the top spots in a jam.

Also, I'd like to rethink the question of how many ratings are too few ratings. I think it was smart to base this threshold on the median number of ratings, so it's not skewed by a small handful of games having way more ratings than everything else. However, I think the median itself is much too high of a threshold- it guarantees that nearly half of the entire field will be penalized, regardless of how evenly the ratings are distributed. It's pretty obvious why this would be a bad thing in the do-not-rank system I'm suggesting, but it doesn't sit right with me to see a game with 9 ratings receive even a small penalty in a jam where the median number of ratings is 10. I'd prefer something like half the median- this would still penalize games that are clearly passed over repeatedly, but if the less-popular half of the games all got a number of ratings that is still pretty close to the median, it wouldn't have to penalize any of them.

Thanks for your time, and I'm interested to hear itch's thoughts on the matter.