Consider the sandbox an artistic choice.
A lot of things you do will affect the story in some way, be it minor or major.
This includes when you do certain events.
Yes, you have main story, but the interpersonal interactions with the girls and bros are important too.