Whenever I manually load a save from the pause menu, the map is pulled up, though I can still advance dialogue. Any save, as long as it's from the Loading... menu.
My temp fix for this I found is to make a new quick save where you're at, start a new game, then quick load. It'll load the quick save without the map up.