You've made a quality mini-golf game and that's to be commended. You also got the time manipulation mechanic working and that's neat too.
Honestly, if you just made it so that time restarts when you take a shot, the game would probably reward a touch of planning and foresight that would make this a winner.