It appears if you earn stardust, and then Prestige, you lose the stardust. You also do not get the stardust back by reaching the same number of Black Hole colliders. Not sure if this is design or not?
Also if "Temporal Prestige" generated 1%/level of your Prestige amount rather than your current points it'd be significantly more useful, and would effectively automate Prestige (which I would assume is the point of the upgrade). Right now if you ever make a jump in atoms, you need to prestige to catch up your Prestige Points or you could be waiting hours.