Hi ! seems like you found the solution yourself, indeed the only way to make a straight collider using tilemap collider its editing the physics shape of each tile, the shapes can match if you take your time and dont rush it. ( Source : Myself ).
You can also use a different collider, using box 2Dcolliders with an invisible sprite, placed on top of the tiles, and then use a composite collider to unify every collider. ( this is easier I think but its not automatic )