In theory, this is a 2d game, the 3d is more for visuals/aesthetics (unless you add 3d elements in the future), I'd recommend taking inspiration from the smooth controls of Riot's TFT, I also had issue with 3d raycasting on a 'board' with an earlier project and their implementation is pretty solid imo
Aside: if you add vertical layers then you could use the scroll wheel to move elements up/down layers