The worst thing uh, well at first I did not understand that fishing was a timing based gameplay so I mashed A button to skip the fishing illustration.
I also disliked the path from the village to the Fish King since it wasn't a linear path, I had to move down several times and it was annoying.