This took me a while to understand what was happening, and most of the time I was just clicking the 5 buttons on the panel. I think the buttons themselves could be a little larger (affordance). I also thought there were some UI elements that could have fit nicely into the rest of the environment. Putting the instructions on the wall was an example of diagetic instruction that could have fit the theme you seem to have been going for. A clock on the wall so that I appreciate that time is progressing would have also helped.

Interesting spin on the theme, it's a good idea! It's is however a bit repetitive, and it could use some music so getting to the end is a bit tedious.  

Curious concept however, I like the vibe and weirdness. :D