I think I got it, you choose to leave the tavern rather than help Burry or talk to Alon and it gives you the choice to take the relic and still be able to go to the sword training.
