the upgrades do keep after switching weapons!
currently, the jam-released version of the game has a bug where most upgrades that relate to the gun don't actually apply **until** you switch guns. this has been fixed in our upcoming update, but for now; you can switch your weapons to apply those upgrades :)