Honestly I just finished my defeat playthrough, and the only curses I ever picked up were the full power Lust and the weakened Greed. It didn't matter in the slightest, as I started literally every single day at 0 willpower. There seems to be some form of cooldown interestingly, as it would only force me to choose one of the sugar daddies every 3-4 days.