Great idea to mix-up the golf formula! If I were to make any suggestions, the controls do feel a little floaty after the first shot as in I am not sure when I'm able to make my next stroke. A simple visual indicator can help.
Additionally, the ball not being at the center of the screen all the time restricted some angles. Maybe the free CineMachine tool in Unity could help you keep the ball in the center while allowing some peeking to see the level. There's actually a Brackey's video on CineMachine.
Still, very solid game, I really liked the aesthetics and again appreciated the added element of destructible obstacles. Great job!