I met the guy in the town square while eavesdropping, and at the end of the conversation he mentioned how the guy who knew what to do frequented the brothel... I have since tried every option in the brothel multiple times (including the girlfriend service) but the quest never continued, did I miss something?