Love the vibes this game gives off. Got a nice eerie atmosphere to it, aided in no small part by the sound and art design working in tandem, so nice work on that!
I do be out here having, like, two brain cells, so I ended up accidentally pressing to restart the game when I meant to swap worlds multiple times, and I also spent a long time staring blankly at the third level and not knowing how to progress. While both of those are most definitely my problem, the former could be a user-experience thing to look into.
The latter I ended up... getting around by not even doing the level and just cheesing it with the fact the door isn't closed when you swap between the worlds and I just rapid-swapped until I was through it. And then proceeded to do the exact same exploit on the fourth level.
I agree with ZCaptainz3 below about the desire to know where a swap can happen, as it just kind of felt like trial and error with rapid-pressing the button until a swap happened. Some extra sound design punch could also work for increasing the difference in feel between the two worlds. Even if you just did, for instance, background music that goes in reverse when you swap.
Overall, neat concept and pretty well-executed!