It was noticeably bugging when walking between rooms, but if you just walked as normally as possible through each room, I found that that mostly solved the issue for me. Kinda reminds me of Binding of Isaac.