Thanks for the feedback! I updated the game and now it stops panning when you move the mouse outside of the game window.