Thank you :)
Looks like what I did wrong was forcefuly adding and trying to open it directly from oculus library, instead of swithing modes on PC.
Had to recalibrate controlers in-game, now it works ok, now I only need to figure out how to open menu without switching to keyboard.