I don't know whether it might be a bug, or a different thing because you beat the caravan, but for me I walked in through the front entrance, like in the screenshot below.
