Haven't played with anyone, but I can see how this could be fun. Clever game design here! Not so many rhythm games encourage team work like this one. And instruments are fun to discover and play, and it's a nice way to balance difficulty.
The fact that the game is playable only with gamepads should be said loud and clear directly on the title screen. It would avoid confusion.