Ok. I recreated the six from this thread as you said and made a research. What I found:

1. The problem occurs "only" (I am not so sure) when the inside shape touches by corner the outside shape. See image 1
2. The problem is not in the direction it is correct (at least in the example of this six)
   1. You would expect (from the algo) the outer circle consists from two contours and the same for six. You expect the contours from circle won't cover six and vice versa
   2. Bugged version consists not from four contours there are more of them. First of all the six is correct but the circle... It covers not only itself but also the six as solid contour and the holes between six and the circle are made with separate contours.
3. I read the article about your algo and it looks like I almost made a bicycle thought my method was based on dots between pixels....

Possible solution:

DISCLAIMER: Maybe everything I say later even not related to real situation and I am just hallucinating...

When I read the code it seemed me a bit strange to see bool in one place:

static function buildPoly(poly:Polygon, colStart:Int, rowStart:Int, z:Bool) {
        var points = poly.points;
        var dir = 3; // ENWS
        var col = colStart, row = rowStart, dirStart = dir;
        inline function add(x:Int, y:Int):Void {
            points.push(new Point(x, y));
        }
        var w = width, h = height, grid = Contour.grid;
        add(col, row);
        var steps = 0;
        while (steps == 0 || (col != colStart || row != rowStart || dir != dirStart)) {
            switch (dir) {
                case 0: { // right (BR corner)
                    if ((col + 1 < w && grid[row][col + 1] > 0) == z) {
                        if ((row + 1 < h && grid[row + 1][col + 1] > 0) == z) {
                            add(col + 1, row + 1);
                            dir = 3;
                        }
                        col += 1;
                    } else {
                        add(col + 1, row + 1);
                        dir = 1;
                    }
                };

IIUC there is happening the phase of creating the outline... And  there you check if some walls in some directions are exist. But in fact we have two different shapes (Do we? I hope I understand) and code should check if there is a wall of exactly the same type but it uses bool so any wall of any shape is considered as wall. Idk I am a fool ¯\_(ツ)_/¯. Anyway I wanted to create my custom similar utility, I even came up with another algo based mentioned above (now I am afraid it won't work)

P.S There won't be normal outline contour for color fonts?