itch.io Community » itch.io » Questions & Support

How does the popular section work? (are we delisted?)

A topic by FrogmanDev created Nov 09, 2023 Views: 2,279 Replies: 39

Viewing posts 1 to 10

FrogmanDev1 year ago (1 edit) (+4)

Hi there,

We just released a demo about a week ago and it was trending at the top of the popular page for multiple tags all week. Twitter was initially driving our views and downloads but itch.io eventually outpaced it via the popular section and some specific tags. Our downloads were actually trending up each day (about 100 per day) but then we suddenly dropped down to almost zero over night. I noticed that our game was no longer anywhere to be found on the popular section under any of the tags we were previously. Is this normal? I did eventually find our game at the very very bottom of the popular page (took 5 minutes to scroll to the VERY bottom). Just trying to figure out if we broke a rule that caused our game to get temporarily suppressed or if that's just how the algo works.

I sent a request to support a couple days ago but still haven't heard back from them so I thought I'd post here incase there was a simple answer. We have some social media stuff we'd like to post but want to hold off until we know for sure there isn't a problem with our account.

Another strange thing, we are currently in the 2nd place spot for "top selling" fps, yet our game is at the very bottom of the popular section which seems strange. I understand this could be totally normal, just wanted to confirm so we can move forward with some things.

Thanks,

FrogmanDev

leafoPinned ReplyAdmin1 year ago (4 edits) (+5)

I can confirm there are no issues with your account. I think this post I made awhile back is still fairly accurate about what goes into our sorting methods: https://itch.io/post/1276094

It’s a complicated system with a lot of inputs that is trying to maximize the needs of over 1 million projects. Our general recommendation is: build something interesting, put effort into the page, and promote it using both the channels we provide (eg. devlogs, building a community, encouraging ratings, building followers, etc.) and external channels (eg. social media, your own communities). It will help our system notice your page.

I think obsessing over itch.io referral view counts for certain days is a risky strategy, as there are many factors that are just outside your control (eg. everyone else on the platform building stuff competing for the same spots on our browse pages). That is why our advice for individual devs is to put in an genuine effort into building an audience. Think of the discovery we provide as a means to amplify your reach.

Here are relevant guide pages for future readers trying to learn more about indexing and eligibility:

Outsider Artisan1 year ago(+4)

This is happening to me and a lot of other devs I’ve reached out to as well. The problem is definitely wide spread, and this is not normal algorithm movement. It’s either a bug or an algorithm change. Either way devs need confirmation.

FrogmanDev1 year ago(+4)

Here's a screen cap of the downloads to get a visual, keep in mind my hypothesis could be entirely wrong.

Outsider Artisan1 year ago (1 edit) (+2)

No, this matches with what I’ve seen and what has been reported over on the Itch discord. I’m actually not in as bad a shape as you, I’ve come down about 50%. Something is definitely going on here. And there were major, massive shakeups in the popular rankings all across the site. While movement is constant in those rankings obviously, in three years on this platform, I’ve never seen games jump dozens of slots in one direction or the other in a single night. Either this was a massive algorithm change or it’s some sort of bug. Maybe both.

FrogmanDev1 year ago(+2)

That's interesting, I have used itch as a player for a long time and never really paid attention to how the ranking worked. It just seems extreme for our game to go from the top of the popular page for multiple days (while downloads are trending up each day) and then suddenly be hundreds of rows down at the very bottom over night. Hopefully someone will chime in with an official answer soon.

FrogmanDev1 year ago (1 edit) (+1)

Update: A prominent YouTube channel just posted a video covering our demo so we are suddenly getting a big spike in views/downloads which seems to have brought us back up onto the popular page. Just some context for whoever ends up looking into this, my original question still stands though.

Tukann1 year ago(+2)

This happened to us as well, we've been looking into it and we can't seem to find any logic into this "culling". I can find plenty of examples of pairs of similar games that were around the top of the "popular" section and overnight one got moved down hundreds of positions while the other one hasn't. It's like a Thanos snap.

redonihunter1 year ago(+1)

I know that the search does rotate. And that the stuff to show is rebuilt regularly. I also know that itch does not tell about how stuff works, to not encourage people to game the system. So do not expect to get an answer. But we can assume that there is a certain amount of rng involved.

And if near 100% of the traffic came from inside itch, a game might not actually be popular. Also in the other direction. If the only traffic comes from outside, but the internal links get ignored, it might be suspicious to the algorithm.

We do not even know how popularity gets weighted, do we? I very much doubt it would be a single number like page visits or download counter.

If they would not have some kind of running average, it would be always the same games on page 1 of browse. So maybe it really does randomly put games on bottom. If they recover naturally, they really are popular.

Of course it could also be some obscure side effect of how the browse index get's rebuilt, if a game get's a major update. Or some anti tampering methods. There do be people that try to fake downloads, ratings and popularity. It is hard to tell, if some game really gone viral, or someone bought clicks.

Or it might actually be very simple. If one does not use infinity scroll, the game in question is right on top of recent page two for tag fps. There have to be some trend setters for lack of a better word. People looking for recent games might be the start of some domino effect. Also, there is a 7 day option for popular games and the game in question is now 9 days old ;-) If the algorithm notices a certain lack of internal interest, it just might not be as popular anymore, according to its rules.

How do I put this. There are those celebrities that are famous for being famous. In my humble opinion, such a source of fame would not sit well with the philosophies of itch, what with the idea of giving all games a spot in the limelight of search results and such. Maybe they do something like that with popular browse.

FrogmanDev1 year ago

Sounds reasonable. Because we have bounced back I am a lot more confident now that we didn't break a policy or rule that put us down at the bottom, must just be how the algorithm does it's thing.

redonihunter1 year ago

If you had broken some important rules, your account would have been banned or something, not just a plummet in browse ranking.

There is a limbo in between, where your stuff is ok, but they delist you anyways. I have seen such an account that would fall under those quality guidelines. But if you are not listed, you do not appear in browse at all.

The old post leafo linked says it clearly, the browse popular algorithm has some parts of real popularity and some parts of boosting for new things mixed in. So erratic behaviour is to be expected.

But if you really were bumped back to last place for some popular tag with thousands of competing games, that does sound like a side effect. Maybe your game looked unavailable due to an update while the index of browse was reshuffled or something silly. Or they lost access data due to their changing servers in the background. There is a thread about that somewhere, that they are migrating some services.

Otherwise there really do be strange numbers. Look at popular games for tag fps. View last day and last week. The first has only 3 games in it. The other 26. Your game is not in there. It is again in the 30 days and overall. Since your game is only 10 days old, it seems strange, that it does not appear in the 7 day category. But, there are a gazillion tags and the popular sort is strange and with small numbers, odd results can happen very easily.

Tukann1 year ago(+1)

Those are a lot of different theories on how the algorithm works, but I don't see how they're relevant here. We're talking about a very drastic change overnight, for numerous developers. My game is more than a year old and I don't remember seeing anything like this happen. I'm talking to affected developers who have been on Itch for 3 years or more and they're saying the same thing.

No Time To PlayModerator1 year ago(+1)

You've been on itch.io for fifteen months and published one game. I've been on itch.io for nine years and published dozens of games. Maybe don't dismiss what old hands tell you.

That said: your game is NSFW. These games have been under increased scrutiny on itch.io since this spring. Our admins say it's nothing, just basic protections against financial fraud. Other people here aren't so sure about that. Either way, the OP just confirmed what I told you on Discord: that organic traffic boosts your ranking. Also, if an affected game can bounce back soon after falling from the top, maybe that means there's no conspiracy. No secret changes of algorithm, or at least nothing permanent. Seems pretty relevant to me.

Outsider Artisan1 year ago(+3)

And here you are again, hunting us down no matter where we try to discuss this to strawman our arguments and ignore any evidence you don’t like. A bounce back from a YT video is definitely good news and a good sign, but it hardly means the algorithm could not have been changed. That’s quite a leap of logic, even for your standards. And you’re the only one trying to put words in our mouth about a conspiracy. I really don’t know why you’re so invested in disproving this when you’re so ill-equipped to do it. If you want to shut us up and prove how dumb we are, go talk to an admin for us.

Airell1 year ago (1 edit) (+3)

Me too. I don't need to say much. My page is dead now... Basically, all the work I did to improve the page, develop and advertising the game was for nothing, it seems.

redonihunter1 year ago

This is barely three weeks. But just like games can go viral, they can go the opposite direction. If I were to take those numbers seriously, there is a downwards trend visible starting from peak of oct 25. You can draw a line from there to now that points downwards, with yet another peak in between on nov 5.

To speculate, your game got a boost somehow. Does not matter from where. But after that boost wore off, the "natural" visitor numbers are more near to 1k than to 4k. There is also a limited number of people visiting itch - compared to a service like steam. It could just be that your potential players in the currently existing user base is exhausted in the meaning of, most people that might be interested in your game already visited your page. Or at least, enough to now have a dwindle in visitor numbers.

I remember your game. I visited your page in the middle of september according to my browser history. Not my cup of tea. But do not ask me, how I found it. I can only tell, it was not via popular browse, as I bascially never use that feature.

As leafo iterated above, you need to advertise your stuff outside of itch. Being on the popular browse page is in itself not enough to stay there.

Airell1 year ago (4 edits) (+3)

Wrong. These numbers have been a constant for about a year, not a "random boost". I invested a lot in art, updated my page, thumb, and I've been advertising constantly and regularly since then. The algorithm literally threw me down, there is no excuse for that.

Be careful saying things like that to devs, you can end offending them. Many work very hard to get what they have. I've always learned that we should respect the hard work of others, never devalue it as "random boosts".

hechelion1 year ago(+2)

You and hundreds of other developers who are also fighting to appear in that space.

Maybe you'll be offended, but when your game stopped appearing there, someone else did and I don't see why another person who is also making an effort couldn't have visibility.

This is not an issue of they harm you because they want to harm you (which are many of the comments I am reading).

That their games lose visibility within the Itch algorithm is because they are being given visibility to other developers, people who are also making an effort and I don't see why you should be given priority over them.
Don't be selfish saying there is no excuse.

I remember that it was always said that you shouldn't trust Itch to give you visibility, if you want your game to be popular, you should go promote yourself in other places and not depend on itch's algorithm.

Outsider Artisan1 year ago(+2)

I can’t speak to Airell, but I’m sure it’s the same for him. I find the idea that I don’t promote my game elsewhere and don’t work my ass off pounding the pavement and that’s why I deserve to have my traffic arbitrarily cut off is offensive. I worked hard for more than three years of consistent quality updates, constant promotion on social media, spent thousands of dollars on artwork for cover and promotional images (in addition to the assets used in my game). But now that the algorithm kicked us off the cliff, everyone ust dismisses the hard work and blood, sweat, and tears as like some type of undeserved snowball effect. Luck has something to do with it, but if success isn’t arbitrary now, and these less popular games deserve more traffic, well it wasn’t arbitrary before either.

The fact is, just looking at the top adult games (my game is an adult game, so that’s the tag I’m familiar with looking through), the top twenty or so games have like six authors. The giga-popular games that have managed to achieve massive notoriety haven’t gone anywhere. What has happened, is games on the second through fifth or so pages got kicked down 100 pages. The pain was not evenly spread.

And sure, we shouldn’t rely on Itch.io to promote our games. But losing 50% of your traffic is losing 50% of your traffic. How is that not supposed to effect devs? It’s like, you can say YouTubers shouldn’t depend on the algorithm, but what does that even mean? Assume all success can be taken? Never try to pour any of the money you’re making back into ongoing development because you may arbitrarily lose hundreds of pages of relevance?

This was a huge change from one day to the next, with non warning, and no apparent way to recover. I’m glad you are able to be flippant and insensitive about it, since that means it didn’t hurt you as bad as it’s gonna hurt some of us, but this is quite serious. Even if logically it means some people are gonna have their games surfaced, leveraging something like a better featured games system that can be curated and used to surface the types of games Itch wants to promote would spread the pain more evenly and avoid these massive drops in traffic many of us are seeing. Likewise, defaulting and refining the New and Popular sorting tab into a “trending” type tab could allow for less extreme changes and less devastating losses.

The fact is, there was almost certainly a way to do this without devastating existing devs. Small drops in engagement spread more evenly across a wider group of devs to lessen the impact individual devs have to bear. I think that would be totally reasonable.

hechelion1 year ago (1 edit) (+1)

Hello.

My response goes to @Airell's comment, so don't consider those words directed at you.

I don't deny that this affects you, but the fact is that it is impossible to please everyone.

Every day there are people uploading games and asking how they can gain exposure on Itch. These people have also worked on their game and they also have bills to pay and that is not talked about.

My question is, why should the algorithm give priority to you and not them?

You say that I don't care what happens to you, but you don't care what happens to those other developers, that's the point I criticize.

In itch there are many things that could be done better, but in the vast majority of the comments I don't read that, for example you say that on the first page there are only 6 developers, that seems unfair to me, but except for your comment, I don't read anyone else talking about that or how to make the algorithm fairer.

Most of what I read (I clarify that it is a generality, I am not pointing at anyone in particular) is a "this harms me and because it harms me (and the rest of the developers who benefit are not interested) it is bad, unfair and Itch is the only one to blame"

That is the attitude I cannot sympathize with.

Advice, do not depend on Itch's algorithm, if you appear popular, take advantage of that boots to create a community, but do not depend or believe that because Itch had you on the first page it will always be that way.

redonihunter1 year ago

My current theory is, that they lost access numbers, possibly somewhat related to this announcement https://itch.io/t/3099694/notice-for-html-game-devs-upcoming-change-to-cdn-domai...

There are 29 games in the popular nsfw text based for 30 days sort. Your's not in it. And there are only 5 games in the 7 day sort. This looks very strange. But I cannot say if it always looks so strange, since I never checked like this. ( The game is around place 110 of 725 for no time constraint.)

Outsider Artisan1 year ago

I mean they changed the algorithm, I used to be in a number of lists and relatively high up, too. I was in the top five overall Twine tagged games (usually floating around number two to number five) for a long time. My other game, Hex, was usually in the top ten or twenty of that list. In “text based + adult” I was also gnerally on the first page/first load, often both games were. They changed the algorithm on the 8th. The list of popular games in the last X days is still informed by the new algorithm, it’s not like a stored historical snapshot or something.

I mean even if you don’t believe me that my game dropped off a cliff, it did. I can see the metrics. I’m not sure why you’d doubt me.

That HTML hosting change has nothing to do with this issue at all, that might just make some players lose their saved games when it happens (and it hasn’t yet), but it won’t effect metrics AFAIK. Even if it does effect metrics, it hasn’t happened yet.

hechelion1 year ago

I thought I saw something strange.

If I search for Tool&Engine "twine" and TAG "Female Female Protagonist" your games appear on the first page.

But if I search by TAG "Twine" and "Female Female Protagonist" your game does not appear.

If I search for "Female Female Protagonist" + "Test based", your two games appear on the first page.

PS: It's not related, but I'm trying your game and it has very good quality, congratulations.

Outsider Artisan1 year ago

Yeah, I actually didn’t tag either of my games with Twine, but it’s listed as the engine. I wanted to use those tags for other stuff. But yeah, with sufficiently specific combinations of tags, my game still shows up high in some lists. And thank you for the kind words, I’m glad you think so.

redonihunter1 year ago

A normal change would not make your games drop so much. However the algorithm now and some time ago worked, it still is supposed to do some ranking based on interactions.

Since there are still games on top of the list, it obviously still sorts them somehow. But there are some massive problems behind the scenes. Servers are changing and I believe somehwere in Asia a whole section was cut off from parts of the hosting servers. There are threads about that.

The algorithm is also supposed to have some methods of preventing cheating. Maybe it is over sensitive to those behind the scenes problems. If they rerouted traffic it might have looked like artificial overinflation of view counts and was ignored, resulting in a lower ranking for some games.

Maybe it is some obscure bug, where only games hosted on a certain load distribution server were affected.

Other than that, a runaway self re-inforcing viral downward trend might be possible due to a small change in the algorithm. Like, you were on edge of page 1 and next day it was sligtly less visits so the algo puts you still further down. Rinse, repeat.

I thought it might have to do with updates, but then I saw a game on page 2 with no updates in a year.

What irks me the most, how the game of OP was apparantly at the bottom of some list. That is why I thought, and still suspect, access numbers were lost or counted as 0 for some time frame.

Airell1 year ago (2 edits) (+1)

Again, I already did. And I do this regularly... For something like 2 years. Huge investment in art, self-promotion on different sites/forums, maintaining a beautiful page, monthly updates, hard work every day to maintain engagement.

It's not selfish to say that hard work needs to be rewarded, it's justice. Justice means giving everyone what is theirs, that is, what you worked hard to deserve.

If everything I said doesn't imply visibility, what would I do? Just sit on my ass in the chair and do nothing? I think the concepts in Itch were quite distorted after this huge mess.

I am very attached to the issue of meritocracy, so take that into account with my words.

redonihunter1 year ago

I did not call it random. Please do not assess any disrespect into my post because of a word I did not use. I only looked at your numbers and why they were high does not matter for that. I literally told so. The somehow includes it being due to your hard work and people noticing it. But as I also explained, people can not notice it twice for the first time, with yours truly as an example.

Also, your new numbers reinforce my viewpoint. You went from 1k to 8k in about a month and plummeted down to 2k a month after that already. I would not call this graph showing constant numbers.

If those numbers could be split into referals, you could be sure, that people previously came from popular browse and now do not.

But even if so. What is cause and what is effect. And any popularity ranking reinforces itself by nature.

Airell1 year ago (2 edits) (+1)

Ohh, I failed to explain this point, I've been doing monthly updates since the beggining. The spikes you see are my advertising on other sites, not just Itch promotion.

In fact, self-promotion helps a little, but still, 90% of my traffic comes from Itch, as well as all the other devs who are suffering from the algorithm change.

Sorry, man. I press on saying, it's kind of obvious... My numbers were stable, and so I lost almost 70% of my traffic overnight. There's not much to say, you sound like you're trying to tell me the sky isn't blue, that the grass is not green, and that Itch isn't actively trying to sabotage us.

redonihunter1 year ago

Your numbers were slowly declining, and with those ups and downs, I would not call that stable.

But the recent plummet probably has other causes. Could still be the opposite of going viral though.

Since you are on page 1/25 of top sellers for html5 nsfw, but only on page 15/137 of popular, something is going on. Page 5/137 might be reasonable. But 15? To be in top sellers, people need to interact with your game first.

My faforite theory now is, that the server your game is hosted on, did not report the access numbers properly, making the algorithm believe your game is less popular wich is a self fulfilling prophecy in the short run.

Airell1 year ago

You deny a fact that is simply evident in front of you, Itch is actively sabotaging us and our numbers have plummeted with the new algorithm. I don't know if you're just trying to troll or something, but anyway... Sorry but this isn't going anywhere

No Time To PlayModerator1 year ago(+1)

Friendly reminder that for almost two days now there's been an official answer pinned to the top of this topic. Obviously some people have seen it, yet the discussion continues to revolve around the same old points, and by now it's just a shouting match. This isn't helpful anymore.

redonihunter1 year ago

The explanation post from leafo, while an explanation for erratic behaviour and ups and downs, is not a sufficient explanation why the game of OP was suddenly sorted last in the browse of popular games. (It only made clear, that it is not something to do with the account)

I did eventually find our game at the very very bottom of the popular page (took 5 minutes to scroll to the VERY bottom)

Something happened. And it is prudent to assume it affected more than one game. And to happen in such a way, to include it, but sort it last, the sortin algorithm must have thought it to have near 0 popularity. Even when allowing for generous adjustments to mix in boosts for new games, this is too inaccurate to be considered normal behaviour. Either the data was wrong or the data processing was faulty.

Airell1 year ago

I understand you're a mod and all, man. I respect your role here. But let's go...

1. You work thousands of hours for two years to create a game, monthly updates, a lot of spending with art assets, self-promotion on different sites, maintain a beautiful page.

2. You get good results, and you are satisfied with it.

3. A year later, the admin decides that you don't deserve the result you have and takes it away from you because others need attention.

Now, my artist and I were materially harmed by this, a concrete thing. Me and many other devs... And yes, unfairly. I just ask for a little empathy from you, and pls try to understand the side of the people who are suffering from the staff's poor decisions.

No Time To PlayModerator1 year ago(+1)

There are over 850K games on itch.io, and who knows how many creators. Even if only 1% of them work as hard as you, that's still a lot of people. I don't see them come here and complain about a feature they were told isn't reliable, demand an official answer, then ignore that answer and insist they must be somehow targeted.

In an earlier post you used the word "meritocracy". You might want to look up why that word is extremely problematic.

redonihunter1 year ago(+1)

the admin decides that you don't deserve the result you have and takes it away from you

You assert human agency. And then base your reasoning on this, as if it were true.

If it were true that someone decided your stuff should not deserve your results, you would have been de-indexed. You are not de-indexed.

While it looks like something did happen that was out of the ordinary and screwed with ranking big time, your game will recover. There is a pool of people looking for games like yours. While that pool might grow slowly as new people discover itch, you still have an area to expand in that can be considered finite. If you do not reach them today, you will reach them tommorrow. But the other way around is also true. If they already checked out your game, why should they do so again.

redonihunter1 year ago(+4)

Even if there were a completely new algorithm building everthing up from scratch in popularity. You not being on top does not translate to you being targeted, it only translates to the other games being more popular according to new rules.

ZanithOne1 year ago (2 edits) (+3)

To be honest, I find it hard to believe that there haven't been any changes to the algorithm or some kind of mass flagging or something. From November 5th to 8th we've lost 70% of our traffic, and I've seen complaints from many developers about the same period.

And ever since we continue to drop in the Popular lists for tags. Around Nov 5th, we were on the fourth row in Popular for our main tag, and today we're in 3000th place in Popular for the same tag, even though the traffic to the page has stabilized.

I understand that the algorithm needs to be opaque to prevent exploitation, but it would be nice to at least know the big no-no's that could cause this kind of fall.

redonihunter1 year ago

I do not think it is any kind of flagging, since, the tags still exist and are populated by the same games. So if there was a "flaggable" problem, why just garble the ranking. There is no benefit. Also, if we assume there are the same number of players browsing, there should be an unexpected increase in views for other games.

The change many speculate about could be a change in weighting of recurring visitors vs new visitors. There is a difference between a user already following you visiting your game page and a new visitor looking at the game. It is a small data sample, but without considering the game in the OP, three of the games complained about here are web games. Those might have lots of recurring visitors.

Anyways, even small changes could have a runaway feedback effect for some games. Especially if some of the fans were located in the regions with the connection problems I read about in other threads.

Tukann1 year ago(+1)

I think the pill would be somehow easier to swallow if everybody was affected equally, but I can't figure a valid explanation for why some games at the top of the "popular" listings were unaffected, despite featuring the same themes, tags, etc as other games that were affected. This does suggest some kind of mass flagging. And if there was, my question is whether this was a one time rebalance that can be reverted based on how popular the game is in the following months, or a permanent flagging, which would hinder the game's visibility no matter how much effort we put into it and how popular it is with users.

redonihunter1 year ago (1 edit) (+1)

Initially I thought it had to do with recent updates. Even the game of OP had an update when it recovered. The games on top of the list of some tags had updates no older than a week or so, when I checked. But there also were too many old games.

But whatever it was, games seem to recover to the adjustment or the bug.

around place 110 of 725

That game now is around place 75 of 726.

I have trouble believing in any kind of flagging, if games can simply recover from it. Also, I do not believe there is any kind of intentional flagging. If they "flag" anything, they de-index it. But I fully believe in the possibility that there could be circumstances in wich the algorithm would decrease the ranking. This includes missing number due to connection problems, adjustment of weighting for recurring users of a web game, random alotment of the boosts it gives away to rotate games, adjustment of weighting of minor/major updates, false triggering of some anti-ranking-cheat, correction of previuosly false attributed view count and of course, bugs.

This topic has been auto-archived and can no longer be posted in because there haven't been any posts in a while.