You can only quickly relaunch the game and not activate it again, or you just remove it from your bonus content folder. It’s generally not advised to have many packs active at the same time.
Deactivating/removing a pack does not delete the saved progress, so you can still come back to all packs later.