I think the story wouldn't flow right if you choose not to interact with some of the girls. I didn't want to seduce the little sister but she does interact with other characters.
It seems like only the teacher you can choose to not have a relationship with (Still playing and enjoying the game though, haven't reached the end yet :D)