Backspace or Esc on the Keyboard. B on the Xbox controller.
You can also press down until you reach the back option
