I was wondering how some UI changes could help with that? I think your UI looks really good, but what I think supported that feeling of being overwhelmed was that you had your arrow keys on the left side and on the right side of the screen the attacks. So you kind of try to focus on the left, but your eyes are somewhat drawn to the right to the attack menu - all while trying to stay in rhythm. Not sure if it would help to have the UI a bit more condensed or make it worse. I ask this out of curiosity, because i wonder if there was a thought process concerning this. Making a good UI for this game seems tough.
Have you considered to do it as in other rhythm games so you can see the inputs way ahead? As in guitar hero, where you see the inputs as a chain and you need to time them right? Here, they just pop up one after another and it makes it feel like a reaction check rather then a rhythm game. I think this might have further helped.
But just to be clear, I think this is a very great entry. Those things might definately worth exploring if you want to keep working on the game, but for a Jam entry, this is really great stuff! But I also think this might actually worth pushing further after the Jam.