Based on the rate all the others improve (1-3 points/day), if charm is raising it is at less than 0.01 points. 400 in game days without any increase, it is not listed in the bug reports. I've even used 2000 gold in temporal beauty, hoping that might help charm raise