You first have to  talk with all swamp village people and report to the Oracle about their behavior. Then  defeat a Gatoroid (those bodybuilder scalies) and  show the head to the Oracle to make the village people friendly. After that you can check the quest board. 

Accept "the elusive husband" quest, talk to the quest giver and then you can interact with the wooden pole to get a boat.