Three fights? Keep going, you've barely even started! Here are some tips:
After four (or five, depending on how many optional ones you do) debates you get a booster pack that expands your options considerably.
Technically you can find cards during freeroam. Some of them are from spending time with important characters (anyone marked with the heart icon), some of them are behind continuing optional investigation objectives (for instance, the receptionist scene in the public affairs office, which you mentioned above), and very occasionally from inspection in the overworld. The latter category is currently the rarest, but that's something I'm seeking to address in future updates.
There certainly are bonuses to picking correct evidence in the middle of a debate, but sometimes your evidence isn't good enough to get the bonus. Depending on how you investigated, you may have entered a debate without the evidence that creates a tangible advantage for yourself. (I'm assuming you're talking about the Lucia debate? This is one example. There's an optional investigation objective you have a narrow window to pursue beforehand that gives you evidence strong enough to end the debate early.)
I'd say you should go until at least the beginning of the Second Layer before making a call on whether the rest of the game has legs. It's certainly a slow-starter (as you mentioned, not unlike Persona 4!) but many of the things you're describing here are not actually absent features!
(As an aside, I should point out that the extent to which you fumble the interaction at the Public Affairs Office is partly dependent on which dialog options you end up choosing. I do like the suggestion of making the player handle it better if the receptionist background has been chosen, though!)