Trippy game, and yes I am very confused! It is a clever mechanic though and a great tech demo I must say, I can imagine it was equally or even more mind bending designing these levels as it is playing them :D
A problem I had was remembering the different paths I had already taken which resulted in me just jumping around and never finding keys. An idea to fix this for stupid people like me could be that you had a trail behind the player that lingered for a bit so I knew which path I already tried recently. Other than that, great job! :)