Ah, I understand. I found the hard way that using a framework is akin to trying to re-invent the wheel. Game engines save you a lot of work over time. Some game engines out there are stripped down and lightweight, maybe consider using one of them. The engine won't stand in your way, and you could add the things you're missing. Best of luck.