They may not have any other targets. Their logic is:
- Roll percentage chance to attack the player (based on Hostility).
- Attack the player directly (if possible)
- Pick a random enemy settlement on the same landmass and attack them.
It could be that you're the only viable target for option 2