Have you perhaps used a code to set relationship levels higher? If yes, same way you can maybe set them lower.
You do not need to start far back, if you did not save often there are autosaves. Try if one of them sets the game to the right footing.
I seem to remember similar situation happened to me. I think it was after using Gallery and reviewing old events. I exited game and continued from previous save.