Setting the xalign to 0.0 seems to have slightly fixed it (the text is actually in the box now)
I'll play around with the xalign and such and I'll hopefully find something that works for sure but at least I know what to mess with now. Thanks!