Thanks so much for playing!
Yes, you were correct to take damage during the Resolve phase when you had no cards in your hand and the Guard hands had higher numbers than you.