Very good starting base! Here are my feedbacks:
- Have the possibility to manually choose where to send (or remove) the workers.
- Add a progress bar for prestige.
- Add a gameplay element like enemies / diseases / building repairs / disasters or others to avoid having a too linear progression curve.
- Have a building responsible for research.
- Have consumable resources like a treasure (+ gold) / a cave (+ random) / a relic (+ research).
- Add fog on the map after a certain level of prestige, this avoids being able to create a building on the opposite side of another. (a building like a "campfire" can help us discover parts of the map)