Would be nice if the fingers worked with holding the buttons rather than toggling. Gets pretty confusing once it's moving fast because of this. I suspect you'd quickly run into keyboard limitations though, as you can only register so many keys being held down at once.
In any case, I like the concept, it's fun and straightforward, and I like that all my fingers are involved and represented by actual fingers.