I'm not the developer, so I don't know for sure, but I think you can just have longer sex, swap more positions, etc. If your stamina is too low, you probably can only have a few positions, and you miss out on the other ones. I don't think the menu where you select the positions looks different with better stamina, but with lower stamina, you don't get to select all positions, because you don't last that long.
Spoiler alert: At one point, they play a game where it seems advantageous to last long, so it's probably a good idea to have good stamina. I wouldn't recommend using all points for stamina, though. My stats are pretty balanced between perversity, stamina, and authority, and I don't care a little less about virility. It might feel a little embarrassing with less virility, because it takes you longer to get hard, but other than that, I didn't experience any disadvantages.