Cheers! Yeah the controls would have to be reconsidered. I think it shouldn't be fully physical, and we need to control the final steps of the placement (making sure that pieces fit in an actual satisfying way).
We have some ideas to make it a bit more puzzly. Having pieces that are interlocked and in your way, so you have to figure out the right order of solving things as well.
Possibly the use of tools to nudge/force pieces into a joint (a wooden hammer and connecting plugs). There's a lot to be explored here. Maybe even making cuts and crafting your own pieces as a solution.