I just did one of the minigames to get one of the items off of me, but then when I went back into the game, I was just gone.

This happened on two separate runs, and I have no idea why this happens.

Seems like when this happens, you're offscreen.  You can potentially get out of it by waiting for an enemy to come toward you (so you know where you are), then trying to walk back to the room.  If you get close enough to touch the exit, you'll move into the next room, and be onscreen again.

...but I usually just reload the game (which puts you back in your "home").