It was a human mistake. I clumbsily pressed, leave, after the first choice (when more options were available).
And when I talked again to the wardrobe, it only gave the option of changing back and leave.
It was not a bug in any sense, it's an instance where the game didn't behaved like my instinct told me it would.
So I brough it up for you to know, If that would happen in an interaction close to the end I would have an issue, but since it only takes 5 seconds to reach the wardrobe it was no issue at all.