You know, speaking as someone who doesn't make games but who plays a fair number of them: sometimes I wonder if game jams are overrated. I see so many entries that are "this mostly works but we didn't have time to add sound or fix that one bug where if you go left three times the entire screen turns upside down and your computer catches fire"--well, OK, then take more time and make the game better! Why are we valuing speed over quality?
Which is to say: I don't think this is a game jam "fail". I think if the game jam made you want to make a game, and rather than rush it, you made it fun (which you did!), you 100% succeeded.