Best source of XP is gemstones. Combine one of each into a rainbow stone, and feed for XP. Crystal Mines, and Farms are the only buildings that really matter long term, but its worth having at least one of everything except research. Once you max research, no need to continue. You passively get research points from invasions and I don't see the point to pushing back invasions late game when that is functionally what you farm as a collector.
Also milking machines can help you unlock most potion recipes without too much effort if you don't want to cheat and look them up.
You can also create a bunch of green gems to feed to mediocre NPCs to give them some decent traits, but its best to do this on unleveled ones because the trait is random, and that means you can get duplicates for stuff you already have. Most common traits are weapon traits, and its pretty easy to get a level 3 weapon skill though before leveling a character. Or you can make a prisoner that is like level 500+ have some traits so they aren't a completely blank slate with tons of levels.