Just follow the main plot for now - you'll need to meet certain characters before you can proceed with vagabonds' questline.
