yes, that's kinda problematic during the Story Mode because Nishy is weakened, luckily that doesn't happen on Free Mode. If you drink your bar's milk, you'll get a temporal speed boost.
If you keep feeling it too slow, I'll suggest you to use a Speed Hack. Makes the experience more fluid