You need to invade a lot of places to prestige. I was on 23 invasions before prestige button said I could do it for 1 golden flag, but it would not let me actually prestige until I invaded 25 places.
I doubt you will see this, but maybe someone else will.