I am curious if it's generally okay to join Jams, that are overlapping with each other, with the same (brand new) game, while also respecting the themes? (or even combining them).

Yes you can, just don’t spam your game into jams with no respect for the rules of each jam just to promote your game. You will be banned from participating in all jams on itch.io if that happens.