Hi, there! Thanks for your question!
A segment is defined by an area confined by the brown lines. So yes, you only change segments when you cross a line. And you can move from the green to the orange in one segment indeed.
If you analyze the map for a bit, you'll find some shortcuts: long segments that get you across a large part of the map. Or you can just go with the flow and experiment a different path on each playthrough! :)
I hope you enjoy the game!