Update 2

Hello guys. After a month of work, I managed to build a prototype of the system I want to develop. I built a webpage, a backend, and collected and processed data for 100k games from itch.io. I also managed to build a recommendation system that can actually recommend something based on user scores for other games, and those recommendations are not completely off. So I’m quite satisfied with the result I managed to get.

The bad thing that the current recommendation system sucks. I mean, it’s so simple because it bases its recommendations on only game genres, so it’s easy to confuse it, when, for example, a player will score a game with the “strategy” genre on 8, and another strategy game on 2.

I wanted to share the first version of the system with the public so people can start using it and give me feedback, but now I see that this has little to no sense, because recommendations will be bad + I need to pay for the servers, and I can’t use the cheapest one (because of data volume and computation load of the system).

So I decided that I will continue with development and will implement the concept of “game theme” so recommendations can start to actually be meaningful, and the recommendation system starts to be deeper. It still will suck, but at least there will be a sense to show it to the public and receive real feedback.

It’s hard for me to estimate how long this game theme thingy will take me to implement, but I think a lot of the time.

I hope to return with the news asap.

Fact #2: From around 100k games I scraped from itch.io, around 5k don’t have any tag related to a known game genre or have a misspelling in the genre name (e.g., “paltformer” vs “platformer”).

Hi, sorry for the late response.

I heard your scepticism regarding using tags as the main data source. Unfortunately, for now, I don’t have a good solution for that problem. Maybe the situation with tags on itch.io will improve one day, we will see.

Thank you for sharing your thoughts on general and sub game genres and confirming that with the otome tag example. I noticed that myself, but now I have confidence that this is something that should be tackled. I will try and see how I can improve the classification algorithm using relations between general and sug genres.

See the computer game genres relation graphs I found, look actually cool:

And after a while, look at what I found! This is very nice from the side of Dexter Yy that they shared that thing with the public. I will definitely use it for the development of my project. Also, thank you for giving me an idea to search for those graphs - found a bunch of cool stuff.

You can’t ping people on Itch. This is not a social media platform to contact and chat with users. You cannot even contact users directly. Nor developers.

Aaa ok, that’s strange, my previous experience with forums told me I can ping a user. Anyway, glad you found this thread by yourself :)

If that is so, your current system’s perspective is very wrong. Breakout and platformers are nothing alike. Hidden objects have nothing to do with visual novels. There is many more such examples.

True. Yesterday, I added more parameters to classify genres in a better way, and I managed to bring down the count of “duplicates” to 13. See the following image how the genres comparison graph looks now:

So thank you for the pin point.

The issue is, if you try to analyze tags on Itch, your system’s data is bound to be faulty. Itch tags are not accurate and you do not know the ranking of relevance, even if they would be accurate.

Yeah, I remember you said it last time in that other thread on the forum, and I also see this now by myself (as I mentioned in the previous post on this thread). For now, I don’t have a way to tackle that problem. From my side, I can only create some kind of tool that helps indie developers to put correct tags or find games with potentially incorrect tags, but this is not the main point of my current project. Also, game developers on itch.io must understand that the better tags they have for their game - the more people will find it and play it.

I am still unsure what you are trying to create. Is your system trying to tell people that like breakout, that they will like platformer games? Or if they like visual novels, that they will like hidden object games? Because, that would fail. Exhibit A is me. I do like some platformers, but I never liked breakout. I like some visual novels, but I do not like hidden object games.

Mmmmm, kinda yes but not. You see, this is the first step in the whole system - compare game genres. Comparing only game genres wouldn’t be a good recommendation system, as you said. I want to go beyond and compare other parameters as well, such as creators, style, mood, etc. Totally, I have planned to have around 13 parameters to describe the game, but for now, I only operate on 1.

Btw, if you like platformers, the current system will likely recommend you another platformer rather than a random breakout game.

You might want to double check popular games on Itch with their respective tags on Steam. Steam tags are ranked by relevancy and relevancy is chosen by users.

Let’s try this.

Backpack Hero, Tags on Itch

Role Playing, Card Game, Strategy

2D, Cute, Deck Building, Dungeon Crawler, Pixel Art, Roguelike, Roguelite, Turn-based

And on Steam. “Popular user-defined tags for this product”

Replay Value, Inventory Management ,Turn-Based, Roguelike Deckbuilder, Deckbuilding, 2D, Fantasy, Roguelike, Strategy, Building,

Choices Matter, Turn-Based Strategy, Roguelite, Casual, Procedural Generation, Singleplayer, Card Battler, Controller, Female Protagonist, Moddable

The developer chose role playing as a genre. Users disagree. It is not even in the list of the 20 most popular tags. Neither is card game, though deckbuilding is used. Nor Pixel Art. Nor Dungeon Crawler. And Steam has a cute tag. But users disagree there too. And yes, the characters are cute, but this is just not relevant, neither is the fact that it is a pixel art game. Curious, isn’t it. Also, the game centers around inventory management, but on Itch no tag indicates such.

I totally agree with this, and I have an idea how to overcome this - when my system can grab data from Steam or other sources, I will merge that data with existing records if there are any. For example, I indexed and put into the database Backpack Hero from itch.io. Then, I grab data from Steam, and while indexing Steam games, and found that I already have Backpack Hero game in my database. So I merge additional/missing data, and I use the same tags when recommending this game, so both itch.io and Stam users benefit from this. The process of merging data is not a trivial thing and will probably require manual review of every merge case, but I hope I will find the best solution if the project survives to that stage.

Update 1

On the first attached image, you can see a graph representing the correlations between different game genres. The closer two genres are, the more similar they are. A black line between two genres means that, from the perspective of my current system, those genres are essentially the same.

In the current system, genres are described by 10 parameters. I may need to review the values for these parameters to better differentiate genres or add more parameters to analyze genres from different perspectives. The graph itself is somewhat strange, for example two very similar genres with only a small difference in weight can sometimes appear far apart for unknown reasons. If you notice any inconsistencies, they might be due to how the graph is visualized. However, I would still appreciate it if you could share your findings so I can double-check them.

On the second attached image, you can see a graph showing the differences between itch.io games from 2013. These differences are calculated based on genre differences. As before, black lines between games indicate that the two games are indistinguishable from the perspective of my system.

Fact #1: A small number of games (fewer than 10) on itch.io from 2013 have no recognizable game genre in their tags. Unfortunately, these games, and any future games with the same issue, will not appear in my recommendation system, as game genre is mandatory for the system to function correctly. If you are a game developer, always include the game genre in your game’s metadata. It’s better to assign multiple genres (if applicable) than to leave it blank.

Overall project progress: The backend is already complete - I can assign scores to games I like or dislike, and the system can already provide reasonable recommendations. Next, I need to create a frontend so interested users can try it themselves, and I also need to collect more game data from itch.io. I estimate this will take me about a month (including frontend development + data collection + and calculations).

Hello @redonihunter, @user22, I’m pinging you here because you were interested in that topic previously in that thread (https://itch.io/t/4748289/itchio-api-for-game-metadata), so I thought you would like to see an update on the thing.

Hi,

A while back, I started a project to build a system that recommends computer games based on individual user preferences. I’m dissatisfied with existing solutions, which often prioritize popularity or generic metrics, and I wanted to create something that delivers truly personalized recommendations, finely tuned to each person’s unique tastes.

I also don’t want to rely on existing generic recommendation engines, so I’m designing a system specifically optimized for games, with the flexibility for users to adjust what matters most to them when choosing their next game.

Right now, I’m focusing on itch.io for a few reasons:

• I want to support indie game developers and help gamers uncover more indie games.
• The current recommendation system on itch.io doesn’t meet my expectations.
• Game data on itch.io is more accessible compared to platforms like Steam.

After a busy period, I finally have more time to dedicate to this project and already have some results to share. That’s why I’ve decided to start this thread - so anyone interested can follow the progress, provide feedback, or share their thoughts.

Thanks :)

👍

Sorry, but I see we are not on the same page there, and I also have no time to go deep into how my idea can provide a value and/or compete with existing platforms. I’m currently planning my things to also be able to answer questions like yours (“Why should I use your system if there are already working systems from big companies?”).

Anyway, thank you for your previous hints and ideas. I will ping you when I am ready to present some “real” things about my recommendation system, and I will be happy to hear your feedback.

Correct, creating a good recommendation system for anything is a very hard task. I’m for sure can’t do that alone, especially because I’m not a data science guy, I have a bit of expertise in gamedev only. What I hope is to be able to create a prototype and see if anyone is interested. If not - I will just move to another idea.

That is highly subjective. One user’s gem is another user’s trash.

Yes, I agree with that point, and that’s what my system tries to do (or at least it will try to do that when it is implemented 😅).

If I understand correctly you want an algorithm to detect underrated games.

Now, I don’t want to create the system that does that. My idea is to create a system that suits every user’s tastes. The original post was about the small part of that system, so I didn’t bother to describe the whole idea because this is not the topic of a discussion. If you are interested in more context, see my reply to @user22 below.

Whats your pov about game tech, its age and status?

Sorry, I don’t understand this question. Do you mean game recommendations systems or gamedev as a whole?

but what happens when the first game is 3D f2p android game and the other is 2D browser furry nsfw game?

I believe this is possible to build a recommendation system that can handle such complexity. It’s hard, but it’s doable. I think the main tactic here is divide-and-conquer - you split the game’s metadata into different, clearly defined characteristics and work with that. From your example, you can describe a few separate parameters:

• dimension (3D or 2D)
• platform (Android native game or PC browser)
• price (F2P or P2P)
• age rating (NSFW or SFW)
• style or whatever you call that parameter (furry or whatever)

After that, you can find a way to correctly classify every game in those categories so you can deterministically compare those games and find their similarity weight.

As the opinion above, itchio gives untied hands for everyone without any harsh rules to follow, combined with its huge content, it’s a big pile of data to analyze.

It will be hard to correctly classify every single tag, but I think it’s possible to classify the most valuable.

The hidden gems you mention, what would you do with them? How would they serve a purpose in your project? Let’s say you found 10 games (hidden gems) which are 99% close to the first game, how would that help your goal? Another example, if all those games are $20 each, i doubt people will happily buy games around.

My current idea is to create a non-biased recommendations system that is based on users’ experience with other games, not on popularity. This is also a good selling point because, from what I know, big recommendation engines like Steam have based their recommendations on popularity and traffic, not on personal experiences. The idea I have is to find not a “good” game but a game that the particular user will like. Some people may enjoy games that are marked as “trash” by the majority and vice versa (some people mark popular games as “trash”).

Hi, thanks a lot for your input. You are correct that this will be nearly impossible to get data about all all games with direct access to DB with no throttling and etc.

I like your idea about accessing only subsets of data, especially when this data is “filtered” by users themselves through the process of reviewing and buying games.

And it already is a bit normalized by only counting paid games that actually sold enough to be in the “top-sellers” list.

Yes, exactly, great idea. I would like my system to find “hidden gems” but right now I have too small amount of time/money to implement such a system properly.

Right now, I’m building a prototype of the system that can find similarity between the game genres. So, for example, we have game X with a game genre “FPS” and Y with a game genre “Third Person Shooter”. Very simple systems will say that those games have nothing in common. I try to build a system that would say “X is 0.7 similar to Y” (based on game genre only) because FPS and TPS have a lot in common, and people who enjoy FPS may enjoy TPS too.

That’s why I need tags. I understand that the data may be incorrect, but for now, I really need a small subset of any sane data to be able to do my calculations.

Anyway, thanks for your help and fresh ideas. Hope to return with some results soon.

Yes. It seems not to contain all the information I need.

Hi everyone,

I’m currently working on a personal project involving a game recommendation system, and I’m hoping to use data from itch.io to train and test my system. I’m running into a few questions about accessing the necessary game metadata and would appreciate any insights you can offer.

Specifically, I’m wondering:

• Direct Database API - beyond the existing JSON API, is there a direct API or access method to the itch.io database itself, that stores game metadata (like tags, descriptions, author, release date, etc.)?

• Game ID List - is there a way to retrieve a comprehensive list of all game IDs on itch.io via an API? I need these IDs to then collect the metadata for each game. Ideally, I’d also like to be able to filter this list by a specific date range (e.g., games released between March 1, 2024, and April 31, 2024).

• API Documentation - the main API endpoint seems to be located at https://api.itch.io/. Is there any official documentation available for this API? I’m finding it difficult to understand the full range of available functionalities.

• API Usage and Scraping Restrictions - are there any specific terms of use or restrictions related to using the itch.io API for data collection or scraping? I want to ensure I’m complying with all the necessary guidelines while gathering data for my project.

Any information, links to documentation, or advice regarding these points would be greatly appreciated. Thank you for your time and assistance!