I don't know if you solved your problem. But I noticed you have to push the direction/movement button towards what you wanted to interact with, and press Z at the same time. Or at least that's what helped me when I was having the same issue.
Hope it helps.