Playing version 0.2.3b, I've encountered twice so far in my playthrough (just got to the VR world) that the BGM doesn't loop, and it seems to be a universal bug that the music stops playing when you roll back, unless you roll all the way back to the frame where the music started and then roll forward again.