I figured it out, apparently messing with the eye sliders is what's changing the shorts. I forgot to remove the shape keys on the shorts.
