I think you are using Euler angles coordinate system; inevitably became gimbal locked, I would consider looking up and using Quaternions which make for a much smoother control system. Definitely hard to wrap brain around but once you code it it makes things so much better!