Oooooh! Now I get it. Sorry!
I was for some reason playing it with a tower defense mindset. As if the fire needed defending...
Meaning any enemy reaching the fire counted as a failure of mine for me.
I did understand that all the yellow things were activatable but even still there were times when you you couldn't kill the enemies before they reached the fire.
Which makes sense. Since you're not supposed to kill them...
I think it wasn't an issue of the game. I'm just stupid sometimes :D