That means your weapon is weak or you didn't touch a strength potions. Some enemies need more than a few hits with a club to go down, but each potion make it easier.
Or you are really bad at mashing one button to escape. Either way there's 3 options how to get past goblins.