Yes, the same thing happens to me. I am using it with v1.17. I thought maybe disabling other mods while using it would help. After disabling Qosmetics I didn't notice it happening anymore, but I will have to experiment a bit more to be sure of that. I have to say though, it's such a cool mod, it's worth me having to restart beat saber over and over. It would be nice if it were fixed though...