this was my initial instinct as well, and the first iteration of the game allowed for pass-through (well, swapping, but basically pass-through). this had the added advantage of making a level generator WAY easier to do as well
unfortunately it also essentially ruins the game... the base matching rules are SLIGHTLY too simple / shallow / naive, and you need the slightly fussy friction of 16-puzzle to make it "complete". otherwise almost every level is disappointingly trivial
game design is such a tightrope! i know there was multiple other ways to introduce the "right" amount of friction tho and im eager to explore those possibilities in future games