The thing is , in general the game give you two options, use charisma or power , when you pick charisma but you dont have enough of it you are going to get refused if not you gone get the same thing like using power just , I suppose, in a more wholesome way , and some time you need to wait etc. Long story short charisma is like using power with extra steps . I am not saying is not worth to use but is for everyone preference, of and just grind it If you need it to progress I suppose, cuz you just need to be inside your house , and idk if there are interaction were you can use only charisma