Good point, I should have just applied a force in the direction of the mouse(for some reason I used linear velocity)