Is the client-server thing free?
I'm haven't looked for a HTTP server recently, the only one I know is Apache Tomcat which is free. There are however also multiplayer functionalities in game engines as well. I know the Unity engine has it (and I bet the Unreal engine has to have it too). The Unity engine and the Unreal engine aren't free in case you make substantial money with your games. But they are free for hobbyists. You'll want to check out your different options of game engines considering you want to do multiplayer.
Some of the software is free.
You still need a server that is reachable over the internet by dns and runs that software. Which for commerical games means renting server capacity and registering a domain name. It's a similar topic as how to run your own website.
If you have a server that runs on one of the player's machines, you need that player to overcome the router issue. That's not that hard, but it's a hassle for casual players. You need some port forwarding and use ddns or rely on finding out the external adress and giving those to the other players.
Maybe there is a free service somewhere for connecting machines to play together. But I guess that's hard to find. Multiplayer games are usually commercial or have local multiplayer. I quick search came up with some adware services and I do not know how trustworthy those are. Also it seemed like they were running the game, not merely connecting the machines.