So... this is awkward...
Like 2 minutes after posting I did some digging and by that I mean I figured out how to work it.
To anyone else having this issue it seems the problem is that the app is inside the app. In order to play, right-click the downloaded file and "Show Package Contents". There you'll find the actual app.