The undo buttons is glitchy when combined with diagonal movement near boxes. If you press two direction buttons at the same time such that you push a box orthogonally adjacent to your starting position., then press undo, the game sometimes reverts the push and your position incorrectly, causing you to wind up in the same space as the box you pushed. This can cheese a number of puzzles involving block pushing in constrained spaces.
Edit: I think it's because the undo only undoes one of the two orthogonal movements performed as part of the diagonal movement in terms of player position, not both. You can see this clearly at the beginning of a level by doing a diagonal move immediately. You can only do one undo, bringing you to a position one space orthogonal from your start position, not two, indicating that the game only records one of the two directions of movement in the undo list.