Just keep holding down the build key after they're fully built, if you have the next tier unlocked you'll start upgrading it.
