Itch's shop system centers around selling access to project pages. A jam is not a project page. They cannot sell entry tickets to a jam.
If you need to filter participants by having them meet certain criteria, like having paid something somewhere, that might be doable somehow.
But I guess your jam will be seen as a scam by many, if you ask for money to even join the jam. There were jams that required the usage of certain software and promised price money and such. Imho it was a mix between scam and advertisement - of course there was crypto involved in the examples I remember.
I searched the internet for "game jam with entry fee" and this very thread appeared on page 1 of the results. That should tell you something. It's just not done. Also Itch sites rank high in relevance for games. I also checked google and the answers from the google ai are garbage (it claims you can filter Itch jams for their entry fee and there are ones that have 200 $ entry fee. It did not understand the difference between a price money and a participation fee. Also I did not even see any price money filter.)
Note, there are jams with entry fees, but those are physical events.