It has been mentioned on the discord - Charisma is currently broken on the dev version - It probably has to deal with how the difficulties interact with stats and custom difficulty and stuff, From what I know, there's some... Crossed wires... that seem to happen
1 (additional) point of charisma will drop the bonus to 1c, I think, Hard to say since I haven't got that build to play around with it.