I've tried doing pairings, but they just wink out at random.
Oh I don't know then :(
Since I wrote my previous message, my clothes unexpectedly changed a few times to what seem to be the default color (the dark red color that clothes have in the UI) when walking around the apartment. Re-selecting the outfit in the wardrobe fixes it. I haven't seen clothes disappear for now.
I'm still early in the story though (I restarted from scratch after a long break) so I may have not yet reached the situations where these issues happen.