Yep, you are absolutely right about everything! This minigame was developed a bit in a rush for the Falling Block Jam 2025, where we got similar feedback.
We plan to improve the game (especially controls) during the current edition of the Improve My Game Jam.
Nice catch with the SFX for placing, now that you pointed it out it sounds very unintuitive. We'll recycle the sound for actually telling the player that the placement was wrong :D
Thanks for the review!