My dumb brain did not figure out that you can hit enemies with yourself for like 10 minutes. Even though 'you are the weapon.'
Indeed the Frost giants won't hit you if you're moving, but you do take just 1 damage for hitting them. Stopping for just a few frames in a large group can mean a lot of damage.