Actually nevermind, I just noticed while testing that I forgot to delete an event that automatically gives you the flag for having started the quest. The only way to fix it (I think) would be to attack the bandits so they're hostile and killing them. Sorry about that.