Loved the game, the really nice piano soundtrack was my favorite part of the game (I love writing music like that myself too!).
It really reminded me of Baba is You (especially the later parts of that game where you have to assemble rule words out of individual letters), which is probably my favorite puzzle game of all time.
Couldn't get past level 4 as I may have managed to bug the game (couldn't work out how to break the bricks, tried what worked in the previous level and spawned no object.)
Like other people said, I can understand why they might have had issues with the controls, but I did not find them "unplayably bad", maybe took a couple of minutes to get used to that's all. I am really happy you implemented the swap with letter mechanic as a nice QOL thing, made the game way less annoying than it could have been.
Definitely would "love" to see this continued (get the pun? :P )!