Damn, it's true that the city popup is frustrating. I think a "press E" interaction would have solved that. With a little more testing this would become apparent. Sorry for missing it! 😅
Glad you like the general idea, tho! In the mines and lighthouses, you can leave people and, when you come back sometime after, you can retrieve them, with a payment made in metal/food (although the equation for it is a little broken haha)