So I was trying to figure out why my project was lagging so bad even though my scene contains 2 objects. Which lead to me literally implementing physics interpolation...
It still didn't work but then I changed my collision shape. I was using single convex collision and I changed it to simplified convex collision and now it works fine.
Oh well! It looks like physics interpolation is recommended anyway so I suppose I will just keep it for now, lol.