Unless Mado adds the ability to build ports on un-pacified areas , I think your just stuck restarting sadly, i don't then there's a way to restart an expedition/map to reselect a starting point.
There's no way to build a port on that same island if you start form the main, as well.
Mado also needs to add some visual indicators on available port building sites or allow them to be built any where that has space imo. 