I was planning a basic settlement system that you build up as you free humans around Aphrodisia. So probably not quite as you're describing--your followers aren't committed to stay with you forever.
Viewing post in [18+] The Restoration of Aphrodisia comments
So something along the line of Raven Rock quest storyline from Morrowind's Blood Moon expansion? I'm looking forward to that, although I'm sad there's not more management/simulation option for basic followers planned.
I do think if you commit to the whole settlement questline, maybe there could be an option where the player leave their followers as settlement guards with some basic need system to keep them working there? This could be a reasonable way for the player to store/collect followers. But obviously you are still the one who is developing the game, so it's your decision to include something like it or not.
I only began playing The Elder Scrolls at the original Oblivion so I'm not too familiar with that. Best case is an isometric view where you can designate plots where certain buildings go and they give bonuses to adjacent buildings, can be upgraded over time, etc. Obviously not a full fledged city builder, but hopefully a little more satisfying (and less annoying) than settlement systems in Spore, Fallout 4, Assassin's Creed 2 (kinda),
Really it all depends how much time I have for development, and seeing as Steam is rejecting this title going into Early Access due to mature content, I may need to trim things down to get it to a "1.0" state sooner.
Perhaps you could build a tavern to recruit new followers and offer a more long-term "storage" plan, but Followers aren't meant to be Pokemon--can't collect them all!
If you played old Oblivion with mods, AFK_Weye was admitted by the creator, directly inspired by the Raven Rock questline from Morrowind Bloodmoon. I'm expecting something alongside AFK_Weye.
But yeah. Don't worry about trying to satisfy other people's want and just focus on what game you are passionate at creating. Because it'll surely be better than something you made out of misguided sense of obligation to others but you aren't passionate about. A lots of my favorite games (quite weird, not cookie cutter genre fitting ones), I never knew I wanted them till I played them.
Speaking of which... not sure if you played one of the 2 Thea games (Thea the Awakening, Thea 2 the Shattering) by Muha Game. But I can definitely see Aphrodisia's setting and lore be great for some kind of mixed/hybrid genre game like Thea. Which use 4X system as a base, but it's mostly a narrative driven game about managing a village (first game)/a migrating tribe (second game) of survivers on a magical post-apocalyptic world. Once again, not asking you to make something like it once you are happy with the current project, but I do want to give a recommendation for the two Thea games if you never heard of them (they are very niche). I feel like you might enjoy them based on the games you made.