Assume I make a cool game and would like to teach others how to make that with a course of my own, whether on some site like Udemy or a video on YouTube, is this kind of self-promotion allowed on Itch.io?
I would check the Creator/Community guidelines. I think youtube channels are fine since I've seen a handful of creators sharing their social media on their projects. A paid course though is a different case, I scanned through the guidelines and didn't see much so probably best to contact support if you don't find a sufficient answer
I guess one could apply common internet sense to this. If you were to encounter a game page and the page would read like an advertisment for something else and only a placeholder to deliver the payload of clickbaiting people to the scamsomething else, that would not be ok.
But if it merely links to your own stuff or stuff related to the project, meh. But do not count on people visiting often. I do not even activate youtube embeds, let alone visit any unneccessary links.
If you have many such links, your profile might be more appropriate, and a short appeal on the project page to visit the profile for tutorials, so as to not clutter your project pages. I seen such project pages. They look awful, consisting of 90% links to other stuff. Devs should consider why visitors are looking at the page. They (and me) are looking at the page to read the description of the game to decide if they want to play the thing, or not.
Oh, and depending on your courses, you might even release them as their own project here. I seen people do that. Videos of course to not work so well on itch, since there is no video player.