The switch is actually instant ability wise but lags behind in the visual side.
You can use Q to switch instantly
