You can beat him if you play your cards right. Just somehow in both my play through I ended up with swords at the most inopportune time!
Maybe have the second sword attack have a bit longer range?
