There is a custom version of the Quake engine, known as "Retroquad", that has a completely new software renderer. The author states that it can be used to create original games, and is GPL licensed. Check it out!.
https://www.moddb.com/engines/retroquad/downloads/retroquad-0100-public-release