Nice horrorpuzzle!
If I understood correctly, then I agree. At first it does seem counter intuitive that you can only interact with objects from the sides.
It's a little sacrifice of immersion in favor of more challenging and compact puzzles.
We thought about having in-game justification for this, I.E Computers looking left/right and crates having some sort of rails that they slide on, so they can only slide to the sides.