I think this is a cool idea but there are a lot of things about the execution that perhaps may need fundamental fixing since the controls end up being very chaotic and difficult to control.
Very neat take on the theme and I think the game would be a lot better with some more polish put into it :D