Mmm it shouldn't restart at all, I would recommend picking a book that has the same color as the flash of light and committing to it ;)
Viewing post in Gardevoir Interactive Pregnancy Puzzle! comments
No, I mean that after the puzzle, there's three button mashes to push out the egg.
But during the third button mash, the book puzzle is active again, and if you successfully do a step of said puzzle in the middle of the button mash, the game goes back to before the first button mash, forcing you to start over.