Finally found the issue and was able to fix the bug! I needed to wipe away the upgrades and recreate them when you prestige!
