thanks for the feedback !
It's because my openxr action map is only configured for touch controllers.
I'm adding this as an issue so I don't forget =)
Btw I'm releasing a pretty major update soon (leveling, random dungeons, inventory, etc.) allowing for a pretty nice coop pve experience (there are more than just 1 gear affix, this is just a screenshot of a test :P) 
