Character movement is relative to the camera, that is true. However, to avoid the situation you described, I created a system where the character’s movement vector does not change when the camera switches. In other words, if you keep holding a movement key and continue moving in a certain direction, that direction will be preserved even if the camera changes. But if you press another key, or release and press again the key you were holding, the character will once again move relative to the currently active camera.
Viewing post in RE Space Game jam comments
Yes, that does help a bit. But the thing is, when the camera switches, there are some cases where I wouldn't keep the button pressed or I press another one too - for example, being dark, I don't always see where is the player character coming from (so, instinctively, I move a bit around using all keys so I can see where I am). Something like that, can't remember really well why I would go back and forth sometimes, until I got used to it.