After fucking him in a fight you'll gain a black heart.. you can still encounter him at all the usual spots but he'll avoid you now.
So you have to go to an earlier savefile where you didn't fuck him or start over again..
