Played it again and while the gifs didn't really help that much I did realize what my issue last time was (it's a wonder what sleep and a break from doing ratings will do for you).
The amount you moved the room in the last room is stored between rooms, and importantly is also limited. So my invisible wall when I got to the first real room was just that I needed to push up against the wall on the right to reset my movement, before going left. It just wasn't super intuitive with that!