The main purpose of Casual is to allow anyone playtesting the game to get all the way to the end, regardless of skill level. Outside of a jam context, I would consider removing or redesigning it, and in hindsight it probably should not be the first setting selected when starting the game.
It's nice to hear from someone who likes games to be hard. Thanks for playing!