Don't worry the ones that don't like Matt are a vocal minority. There's tons of player that like Matt, but that's the benefit of having choice.
For the Bus scene, you have to date Kate and be with Matt. When you go to the Zoo for a Date with Kate he'll show up on the bus.
With Damien you have to ignore Damien in the very first meeting where he's being bullied by Alex then go down the Matt route.
As one of the Matt enjoyers, I just want to say that he is my favorite. A lot of times, when playing as a female protag in h-games, the player either has yuri content or ugly bastard-type stuff. And while no shaming here, I'm not into either; having Matt as an option is really refreshing for me :)