It worked surprisingly well, considering I put this together in only 48h (only thing missing is reconnecting in case a peer connection breaks).
You understood correctly, peers connect directly to each other. The small signalling server is only used to "discover" the other peers and exchange connections information. WebRTC uses STUN/TURN in order to connect.
STUN basically does "hole punching" to traverse through NATs to a target device. Basically a local device can connect to a STUN server to create a hole and be available from the outside (think of it kinda like an automatic port-forwarding of a router).
If that fails, WebRTC will fall back to TURN. This is basically a relay server that clients can use in case that STUN failed for their particular network setup. This does introduce slightly more latency as a direct connection, but it still is super fast as packets can be basically relayed 1:1 without any overhead.
Regarding global IPs: With the context from above, yes, it uses the public IP in the case of STUN being successfull. Otherwise the traffic goes to the TURN server first before being forwarded.