That's weird, I'm not sure what the problem could be. You shouldn't need to install Ren'Py.
Did you extract the files first? When you do, in the extracted folder, the application should have an icon with a picture from the game. There's DoNOTTakeThisCatHome (for 64 bit) and DoNOTTakeThisCatHome-32 (for 32 bit). You don't click on the .PY or .SH files.