The idea is very creative, but I think the execution is a little lacking. I find the lack of feedback and game feel to be a big issue. It isn't clear when I can attack, and when I attack, my attacks feel underwhelming. Add some effects, some sound to enemy hit/death, make it feel like the player is doing something correct.
Also would be nice to have a clear goal in a jam game (e.g. find 4 pillars). I wondered about, met 4 different enemies, and then I got disinterested.
Try not to include elements that are "nice to have", but ultimately don't matter to the core of your jam game. Inventory items are nice to have, but your time and energy would've been better spent doing something else.
Sorry if I sound too harsh, but I think the concept you guys came up with is brilliant, and I hope that some of my feedback here can help you make better jam games in the future :)