I don't have much experience in designing game, but I have to admit that using a gravity change mechanic in itself is challenging. Your game also allows walking on walls and changing the polarity on those as well, so it's an extra dimension of complexity. As a player, I find it confusing when left / right are inverted. You could try keeping absolute left / right even when changing polarity from floor to ceiling and viceversa. For the lateral walls, it's a little different, but similar in a way: when going from floor to right wall, my brain could easily understand that right key goes up, but after switching polarity and landing on the left wall, somehow it's very hard / unnatural to switch the expectation that right key goes down. However, this might prove tricky to handle when going from floor to wall, and viceversa.