I feel like the rhythm game playing at the same time as the battle takes away from from the fight, i think it would be better if either:
a) had the rhythm game overlaid over character like this mock up I just did
b) have the mini game play in waves between where the battle stops and it just powers up the next hit that will happen
the battles felt to long as well, your early fights need to happen faster and then build up in to longer fights as you get stronger and so do the opponents
hope this feed back is helpful