Good news, it's *almost* perfect now with FFS partition! Just a couple of small stutters in the whole video (one in the middle, another in the last few seconds), but it's far better now with PAL. And it's perfect for PAL systems like mine :)
In PFS3AIO partition it's now similar to what the NTSC version was with FFS, meaning it has some stuttering here and there but is watchable.
I tried 8khz and back to 11khz, no difference in performance! The bottleneck is elsewhere now. I do have 4MB fastram and I think it's preloading the audio because it takes about 5 seconds to load and start playing. Since 11khz has much better quality in practice, I'll stick to it.
I presume this is as good as it gets without significant changes to the engine. But I wouldn't worry too much, as long as people have a FFS partition things work really well.
Thank you again so much for making this possible! Looking forward to showing this off around here :)