It's because the feature is already kind of confusing for players and adding more specifics on how it works and what doesn't work is too much. Since you can't be jealous over a neighbor, you can't use the jealousy feature at all. That's the most simple and easy to understand option for the system.