In one of my other games, I did something similar to what you suggested (rather than a menu, it was a grid of gameplay elements that could be navigated with mouse or keyboard), and a blind player ended up with focus unexpectedly moving around because they unknowingly had the mouse cursor over the playspace.
Thanks for sharing this experience, I actually didn't quite think it through. I will need to pay more attention to how other games do it. It still feels a bit off in Cloud Courier main menu for me personally but now I am not sure how exactly it should be.