You will also need to recompile the source code with your custom formula. In main.cpp there's a comment "//All fractal equations" and under it you can add yours or modify an existing one depending on what you did.
Could you tell me how to access the main.cpp file? In the downloaded folder there's only the exe file and the glsl file. (Along with the copywrite stuff).