It's a custom engine.  As it says on the github source, "We haven't done a lot of 3D development so much of this is not very optimized, it's coded in libraries mostly intended for 2D games, and we put it all together very, very quickly."  If you want the details go there and look at the source.