This is basically the video game equivalent of trying to pat your head and rub your belly at the same time, haha. Very creative game and the cartoony background art is charming. Unfortunately, I'm not too good at rhythm games and this is basically two rhythm game in one.
I think it would help if the game was easier at first to ease the player into the mechanics. Like at first the swimmer has less enemies to dodge and the arrow clicking has single inputs, but then more enemies and double inputs could come in later. I found it hard to get into a rhythm because it was too hard at first.
Overall, nice job and creative idea!