So I have noticed, that when I talk to the girls in the game, Even at 40%, they accept my invitation to bring them home. And the percentage of sleeping with them is roughly the same. But every time I click the action, they say no. Is the percentage based on the chance of them saying yes, or do I have to get 100% with them before doing anything?

That 40% is how likely it is for them to say yes. You don't need to have them at 100% for them to accept.

The percentage is based on the affection stat. If you max out their affection, the changes will always be 100%. But in theory you can invite a girl with a 2% change and if you are very lucky, she can accept (same with sex).

The way the sex chance is calculated uses another hidden modifier tho, so it's always lower than the chance of accepting coming with you.