Reversing into the pit should've been the only "invisible wall". I did run into a similar issue though, where sometimes the player would suddenly slow when it shouldn't, but I wasn't able to reproduce it.
With the pit, you're supposed to drive in from the left side (where the sign is pointing), and it should assist in steering if you're pressing A or D. The fuel used to be so much more punishing, but my younger brother despised it so I made the penalty much less severe.
Thanks for playing and for the feedback!