In the real world, water often takes a long while to cool down at night. I could explain the science behind it, but to keep my comment short, I'd still like to say I agree with your idea. The water would be cold enough to start hypothermia about two to four hours after the sun goes down (translated to game time, approximately 1 and 1/2 to 3 minutes after the sun goes down).
Hypothermia itself should also make the player a bit hungrier for a while even after they get out. Perhaps a 3 to 4 minute hunger debuff? The torches and other fire-utilizing items would help the debuff go away faster.