Yeah, this one is a little bit too abstract, even for me.
The goal/purpose of the game is not made clear when playing, and without your comment below neither is how to play!
Clicking the gems in order and pressing the right button gives the same result as just clicking the right button and no gems -- or really, the same result as not clicking anything at all. It would be more enjoyable if there was some kind of visual or audio feedback. Perhaps a quick "ding!" or a "womp womp" to show if you got the sequence right or not would do wonders for this game. That and an in-game explanation on what numbers correspond to what gems.
As it is, I was not sure if anything was happening when I was playing. I do think what is present is neat, now that I know how to play.