I really loved the atmosphere ... the sound of the waves, spray or water, and the rolling effect. You really feel like you are out of your depth.
For me it might have actually been too realistic in terms of combat. I don't think I was even able to hit the first boss. If I did, I did not see any indication.
I would suggest starting with something a little easier, maybe taking out a few sharks with one shot each, to give the player some positive feedback and a feel for the weapons and movement before pitting them against a giant boss whale. Just a suggestion.
I will keep going ... feel free to ignore ... If you've played Legend of Zelda: Windwaker, I really loved the fights with the giant squids. They were much easier of course, but they put your boat into a whirlpool that spins around the squid. This looks terrifying but actually makes it easier, because you no longer have to control the boat. I'm not sure if you had a time limit in that case...
There was also a great boat/monster combat in Resident Evil 4, but I can't remember exactly what I liked about it.
Finally, Call of Cthulhu: Dark Corners of the Earth has a fight against Dagon, where you are on a ship. You have to grab hold of the railings when he shakes the boat, then you have a few seconds to get a shot at him with the on-board cannon. That was probably my favorite boat/monster fight ever.
Again, these are just ideas I am throwing at you. I do really love the idea and how well you've made this!