X is actually handling ALL of their problems without you, they're cutting you out for a very good reason. I wouldn't mistake their personal trauma and their very real concerns about the Crown as them just "being an idiot"? X is one of the smartest characters in the whole game, the Crown literally would be dead without them. At this point in time X has no obligations or loyalty towards your Crown, because you haven't done anything to earn it.
From all your comments you've posted on this page, it sounds like you don't understand X's character arc whatsoever, and it seems X doesn't like your character either since you have a low relationship with them. I would recommend trying another LI lol.