Saves stay just fine. I know the past couple updates had a brief problem where the save wouldn't load the new update, but that's only for a couple of them.
The old apk or zip can totally be deleted after the game info is installed. at least from my experience.