Yes - you could change the colour at any time throughout the game by going back to that bedroom and changing to another pair of underwear.
In any case, it's great to hear you ended up liking the game despite the occasional frustrating aspect - the feedback will hopefully help make anything I work on next more streamlined and less irritating.