I now figured out the speed didn't increase properly, and it feels completely different now that I fixed it (in my working build). I guess that's what happens with badly implemented code.