The jellyfish-looking things that pop out of new holes and then fly down: it feels really cheap that spikes don't hurt them. Sometimes they appear so suddenly that I can't dodge to the side, and I've thought "well at least I have my spikes" then nope, they hit me and I lose all my spikes. Suuuuper frustrating. I'd feel less frustrated if they telegraphed their arrival sooner, or they moved more slowly.
And here's a random thought: I agree that the "last heart" noise is annoying, but it's serving an important purpose. Consider adding a new layer to the soundtrack instead? Something heartbeat-y? This is a completely unsolicited idea but warning noises are rarely not annoying. (The "you have 30 seconds left" alarm in Katamari Damacy struck me as surprisingly bad and that game came out like 10 years ago.)
I was surprised that tumbleweeds were an insta-kill. I discovered this with four spikes and three hearts, and that sudden death suuuuucked. (Roguelikes feel best when you feel like every death is a learning experience and not a cheap death, so I'm trying to share when a death feels "cheap.") If they were anything else that better telegraphed insta-death, (say, a giant boulder) I'd be less frustrated, but for now they look like big fluffy clouds that should do a heart's worth of damage and pop with one spike. Also: holy fuck there are so many tumbleweeds. Between them and the holes, more often than not it's about waiting a looong time time to see an opening, since most of the time they appear to be presenting an unwinnable configuration of tumbleweeds effin' up your business.
(I am loving the game so far, by the way. It can get difficult right out of the gate, and I appreciate that.)