Hey!
That's awesome, thank you so much for finding this... Unity UI has so many specifics to it, I probably wouldn't have found this myself. I'll make is so that always gets called when Rebuild() gets called. It seems to solve it!
Thank you!