That's not entirely true. Spending the festival with Violet does help, but it isn't necessary. Her accepting or refusing depends on your choices during her whole storyline by that point.
Oh, that's odd. When I failed getting the date with Violet, I changed a lot of the dialogue choices in repeat runs (Tho' I didn't pick the ones which just irked me), with the only actual change resulting from spending the time after the festival with Violet, then going the route of asking her out.