Controller (XBox and Switch) work fine with this Plugin.
Yeah I found out the issue. I had to turn off the STEAM Input in the properties menu. After I did that, I was able to use my controller again.