You can alter the scaling of every stat (even Ego) on the Scales menu located in the Character tab (the right one at the bottom, also available to other characters that follow you)
Changing the character's visual scaling had no effect on this bug.
