You technically gain more, as each time it passes through the shifter it converts more light. So by chaining 3 together you convert all light, including blue, negating the 10% efficiency loss.
You should split the infrared light energy evenly between the two blue shifters, and also use convex lenses to push more energy directly into the blue and red shifters.
I used your idea of reflecting the light back towards the other side, and it let me make this!