After testing, I think it's better to zero the score only if the angle is wrong, but not if the distance is wrong.