yeah, being high enough level to avoid the one shots is important - check the wolven vendor shop to buy a piece of jewelry if you haven't already - then go pay a visit to knave and have a chat! that might help you out a bit.
Ok I beat him : D I resetted and re invested stats into magic and speed, while for hunter I upgraded both physicality and spirit hunter. This time I beat him at