In dub mode, there are three main issues:
- The audio processing can freeze while playing the original clip, meaning that the waveform graph is out of sync with the actual audio. This makes it hard to judge when to speak for an accurately timed dub.
- I would suggest processing the audio waveform for the original clips as soon as they are loaded, and then simply reveal the waveform image, instead of actually doing the waveform rendering in real-time.
- Similarly, during recording, the waveform can freeze, which can be off-putting for timing.
- My suggestion for this would be to draw a more simple graph that takes fewer draw calls so that it's less likely to struggle. Also pause any unnecessary background file I/O processes that might cause a freeze during recording.
- This is the most egregious - when playing back the dub, the video can freeze, and each freeze puts it further and further out of sync with the audio. I have had video clips end several seconds before the final line was spoken. For a game where synchronization is the entire point, having this during the final result really kills the vibe.
- I would suggest pre-loading all voice clips into memory (and possibly mix them into a single track/buffer) before the playback begins , to avoid needing file I/O during playback and causing a freeze.
Hopefully this is useful. If you would like (free) help with testing, custom audio code etc, feel free to get in touch, I'm something of an audio-visual programmer myself
