Concept is cool. Very original.
The added staggering is a bit annoying given the mouse movement is already not easy, if you reall y want to implement stagger add it later, specific for an enemy, and progressively harder.