When upgrading units, it should also upgrade the units that are currently spawned. Right now you have to purposely kill your archer units to "update" them. This should not be an intended mechanic.
I agree with you! The dungeon already does this, I will unify it for the next update so your units can upgrade as you progress.