I want to say that if your question was as to why color fonts don’t use the contour algorithm, that’s why.
Glyphs are not traced together, each one is converted to an individual grid and then traced.
I do not have the mental capacity to rate (or implement to test) your solutions at the moment.