As others said the mechanic is confusing, even after reading the description and comments and retrying to match stuff the best I can it was never doing what I expected.
So an interesting idea, just needs more work, possibly more then a game jam would allow but it could be something cool if worked on further.