Thank you very much for the kind comments.
That glitch turns out to be the infamous "corner collider glitch" of Unity :( Looking for a solution. It's a tough physics problem and it haunts many games. I would love to hear ideas for the fix. This is a hot topic on Unity forums but I couldn't find a clean enough solution. Sadly, the unclean solution that I reluctantly implemented is: out of bounds colliders.
I'll definitely improve the mouse look feature, thanks for pointing it out.