I think it may be a recurring issue with the unity input system. We did a version jump, and with it new versions of the software that runs the controller inputs, a version that many projects are having performance issues with. It is something that's actively on my plate, but I've yet to find a solution, seemingly the only workaround is using a mouse and keyboard. I will ping this thread when I've solved it and so sorry that's the best answer I can give at the moment.