Hey, this is a very long time to respond, I think I just forgot that this was asked.
The way I see it: - modifiers are parts of you that you don't like, or disagree with, or things you are *bad* at. I think that's up to you to decide. All that matters is that you will roll lower using them haha