I arrived at the docks, and now what do I do?
Find the swordsman in tavern to get the key to the cliffside temple.
(I'm assuming you've spoken to the hermit already.)