It might just be the controls that people were having a hard time with, or I made the recoil upgrade a bit too powerful. Either way, I made an "improved" version of the game now with the option to change the direction of the bullet, since I wanted to see if it felt better to control with the bullet pointed towards the mouse.
It's really hard to tell if I messed up the movement mechanics or just the controls.