Where did you leave your ship? Once you find it, get on it and press the little button in the bottom-right corner to sail back to town.