itch.io Spring Selects Series A
On Sale: GamesAssetsToolsTabletopComics
Indie game storeFree gamesFun gamesHorror games
Game developmentAssetsComics
SalesBundles
Jobs
Tags

Amos

703
Posts
27
Topics
394
Followers
447
Following
A member registered Jun 26, 2014 · View creator page →

Creator of

Recent community posts

(1 edit)

See https://github.com/itchio/ox/issues/3

butler v15.19.0 is out with this fix.

As usual, the itch app will automatically update to it over time, but you can force the update by going to Preferences (bottom-left button in sidebar), scrolling down to “Advanced”, and clicking “Check for updates”

After that, it should look like this:

image

(Please report back with whether or not this fixed your issue on NTFS, ZFS and eCryptFS filesystems)

I’ve just released itch-setup v1.24.0, which no longer uses CoIncrementMTAUsage and should work on Windows 7.

You can download the latest version from https://itch.io/app, as usual.

Please report back whether it does or doesn’t work for you on Windows 7. (If you’ve gotten itch installed in the meantime - it’ll just verify and repair your existing install, you won’t lose any data).

I’ve just released itch-setup v1.24.0, which no longer uses CoIncrementMTAUsage and should work on Windows 7.

You can download the latest version from https://itch.io/app, as usual.

Please report back whether it does or doesn’t work for you on Windows 7. (If you’ve gotten itch installed in the meantime - it’ll just verify and repair your existing install, you won’t lose any data).

I’ve just released itch-setup v1.24.0, which no longer uses CoIncrementMTAUsage and should work on Windows 7.

You can download the latest version from https://itch.io/app, as usual.

Please report back whether it does or doesn’t work for you on Windows 7. (If you’ve gotten itch installed in the meantime - it’ll just verify and repair your existing install, you won’t lose any data).

(1 edit)

I’ve just released itch-setup v1.24.0, which no longer uses CoIncrementMTAUsage and should work on Windows 7.

You can download the latest version from https://itch.io/app, as usual.

Please report back whether it does or doesn’t work for you on Windows 7. (If you’ve gotten itch installed in the meantime - it’ll just verify and repair your existing install, you won’t lose any data).

I’ve found the bug, fixed it, and released butler v15.18.0.

Memory usage is back to normal:

Everyone will eventually be upgraded to butler v15.18.0, but if you want to force the update now, you can do so from Preferences => Advanced => Components => Check for updates

itch app maintainer here, that’s my bug (well, one of the dependencies, but let’s just say it just became my bug), I’m looking into it at the moment.

Thanks for reporting back!

(1 edit)

Hey all,

Sorry for the trouble. This was a legitimate bug, and it only affected 32-bit versions of the installer, that’s what I didn’t catch it earlier.

I just released itch-setup v1.23.0, you can download it from https://itch.io/app as usual. Make sure you run the version you just downloaded, and not the other one you have on disk right now that doesn’t work!

Thanks for the log. Still, it’s kinda hard to tell what’s going on.. I did notice a bunch of network errors when the app was doing its self-update check, and when downloading locales…

Do you have any third-party antivirus running?

Type Dreams is distributed as a .zip file, so the “good” strategy is used - the .zip is never stored on disk, and instead extracted on the fly.

In that case, you’d see all the files on disk as soon as the download starts. They’re just preallocated at the start of the single download+extract phase. (That should answer 3)

For 2, you’d only see a second progress bar if the format was something other than .zip, so not the case here.

For 1, well, if you had a progress bar restart from zero while downloading Type Dreams, something else went wrong, and I might be interested in looking at your app log: you can find it in “Preferences” => “Advanced” => “Open app log”, from that window you can type “Open folder” to locate the file. Or you might be able to diagnose just from reading the messages :)

Hey, I maintain the itch desktop app.

It does actually resume downloads. Here’s what it’s supposed to look like:

https://youtu.be/g3WyjRo4oj0

However, depending on the contents of the upload, the app has a few different strategies.

For formats like .zip, which is what you get if developers use our tool for uploading games, the situation is ideal. The app actually manages to extract files while downloading - so the .zip file is never actually stored on disk, which saves on disk I/O and disk space.

The tech involved is actually pretty neat - the app stores the state of the zip decompressor (I just call those “checkpoints”) on disk every few seconds, which means it can resume even if you quit the app and restart it, or if your computer is shut down.

For formats like .7z and .rar, “streaming decompression” is impractical due to how those formats are designed. The best we could do is save checkpoints after every entry in the archive. And some games, like UE4 games, typically ship one gigantic .pak file and then a few small files. If downloading was paused in the middle of extracting the big .pak file, then that progress would be lost.

So we don’t do that. For those formats, like .7z and .rar, we just download the entire archive file to disk first. And then we decompress from the disk, to the disk. Which means you see the progress bar go from 0 to 100 twice. Once when it’s downloading, and the second time when it’s extracting.

So - as long as everything is working as expected - the only time you should see a progress bar restart from zero when you resume it, is if it was 1) a .7z/.rar file 2) the file was fully downloaded to disk already 3) it was in the middle of extracting it.

Even then, we do save checkpoints while extracting from disk to disk - to avoid re-extracting the same files over again on resume. So as an additional condition, it would have to be an upload from an UE4 game, or another game that has is mostly one huge file.

So I guess my question is: what were you trying to install? I’ll definitely want to try and see if I can reproduce the problem locally.

It’s not possible to yank a build at the moment. Although, I can see the use case for that and I’ll try to think of a way to make it happen.

In the short term though, nope, sorry.

You’re using an older version of the app, you can get the latest here: https://itch.io/app

VC 2019 is now available as a prereq, and it should also automatically detected when a game that needs it is first run in the app.

Hey there, glad it worked for you!

The Go folks are still curious as to exactly what caused the problem in the first place.

In particular, that command:

GODEBUG=x509roots=1 go test -v -run TestSystemRoots crypto/x509

You’ll need to download the Go language and install it first, here’s a direct link for macOS: https://dl.google.com/go/go1.13.5.darwin-amd64.pkg

After that, you might need to open a new terminal or even log out / log in again, see https://golang.org/doc/install#macos

The fix I shipped is just a workaround, hopefully the root issue can be resolved!

Thanks again for getting back to me every time :)

I’m back with some more information from the Go team, and some more questions:

  • Are you behind any kind of proxy?
  • Can you access https://api.itch.io/ with Google Chrome? With Firefox? With curl? (curl https://api.itch.io/ in the command-line). What certificate chain do you see in Google Chrome & Firefox?
  • Can you install the latest Go from https://golang.org/ and run GODEBUG=x509roots=1 go test -v -run TestSystemRoots crypto/x509 ?

Please respond in as much detail as you can!

Update: I had another idea (just shipping a CA bundle with butler - only for macOS though), and I just shipped butler v15.7.3, can you try again?

It should upgrade butler when the itch app starts, you can always look at ~/Library/Application Support/itch/broth to make sure it grabbed v15.7.3

About the certificate - it’s the one for api.itch.io, here’s what Chrome tells me on Windows (gotten from visiting https://api.itch.io/profile and clicking the lock to the left of the address bar, choosing “Certificate (Valid)”, which opens the default Windows dialog for certificates):

/Applications and ~/Applications are both fine places to put an app bundle. I made itch-setup install to the latter so that it doesn’t need Administrator access (I strongly believe installing games should never require Administrator access), so that’s not the problem.

Except for you, I feel like itch really doesn’t give a crap that I can’t access games I’ve paid for.

We try to provide the app to make it easier to play games, but there’s always “downloading directly from the website” as a fallback - you can do some from the download pages for any games you’ve bought! So I wouldn’t say that statement is accurate.

When directly downloading, though, you are going to run into other issues - many games aren’t signed, let alone notarized, so you’ll have to right click -> Open, but those are decisions Apple made that affect a lot of developers, and not something we personally have control over

it’s one of the many reasons I bloody hate game website apps. Half the time, they don’t work. Plus, they take up a crapload of space to basically function as a folder and connection to the website (mining my gaming habits in the bargain). I have four different game website apps and they all suck.

I feel you. I’m the sole maintainer of the itch app, and believe me when I say I’m trying - hard - to make it lighter and faster. I’m also not fond of our competitors’ apps either.

The problem you’re encountering here is quite fundamental - it’s not some small feature that doesn’t work, it’s that the app can’t verify the certificate of the itch.io API server on your machine. According to the Go issue I linked earlier, this may be caused by “enabling cgo” - but that’s not something I can disable, because butler (which powers the core features of the app - fetching your library, installing/updating/configuring/launching games) is written in Go and uses a handful of C libraries, like sqlite, a brotli compressor, bindings to 7-zip, etc.

If I had access to a mac that had the same issue I might be able to find a workaround, but short of that, I’m really not sure what to do. Disabling certificate validation is not really an option, because that would allow anyone to snoop at traffic between you and itch.io.

I see that there is another issue opened on the Go repository more recently (27 days ago) with the same error message, with or without cgo enabled: https://github.com/golang/go/issues/35631 - unfortunately there hasn’t been much activity there :(

In any case, I can tell you that:

  • This is not a problem with Install itch.app or itch.app - they both perform exactly as expected.
  • This is an issue with the “butler” component that itch.app downloads and extracts (successfully on your machine)
  • This is not a bug in “butler” itself, but a bug of the Go language standard library on specific configurations on macOS. I’m not sure what’s going on there, and neither are the Go developers apparently.

That last issue mentions that setting certificate trust settings to “Always Trust” fixes it for them, which makes me wonder, were all the certificates set to “Always Trust” ?

I just looked up the certificate chain again and it seems the root is “USERTrust RSA Certification Authority”, which for me on macOS Catalina is set to “Use System Defaults”:

The first Go issue I linked gives a command to inspect certificates, which I just used on my Catalina machine, and it gives:

$ security find-certificate -c "USERTrust RSA Certification Authority" -a ~/Library/Keychains/login.keychain /Library/Keychains/System.keychain /System/Library/Keychains/SystemRootCertificates.keychain

keychain: "/System/Library/Keychains/SystemRootCertificates.keychain"
version: 256
class: 0x80001000 
attributes:
    "alis"<blob>="USERTrust RSA Certification Authority"
    "cenc"<uint32>=0x00000003 
    "ctyp"<uint32>=0x00000001 
    "hpky"<blob>=0x5379BF5AAA2B4ACF5480E1D89BC09DF2B20366CB  "Sy\277Z\252+J\317T\200\341\330\233\300\235\362\262\003f\313"
    "issu"<blob>=0x308188310B3009060355040613025553311330110603550408130A4E4557204A4552534559311430120603550407130B4A45525345592043495459311E301C060355040A131554484520555345525452555354204E4554574F524B312E302C06035504031325555345525452555354205253412043455254494649434154494F4E20415554484F52495459  "0\201\2101\0130\011\006\003U\004\006\023\002US1\0230\021\006\003U\004\010\023\012NEW JERSEY1\0240\022\006\003U\004\007\023\013JERSEY CITY1\0360\034\006\003U\004\012\023\025THE USERTRUST NETWORK1.0,\006\003U\004\003\023%USERTRUST RSA CERTIFICATION AUTHORITY"
    "labl"<blob>="USERTrust RSA Certification Authority"
    "skid"<blob>=0x5379BF5AAA2B4ACF5480E1D89BC09DF2B20366CB  "Sy\277Z\252+J\317T\200\341\330\233\300\235\362\262\003f\313"
    "snbr"<blob>=0x01FD6D30FCA3CA51A81BBC640E35032D  "\001\375m0\374\243\312Q\250\033\274d\0165\003-"
    "subj"<blob>=0x308188310B3009060355040613025553311330110603550408130A4E4557204A4552534559311430120603550407130B4A45525345592043495459311E301C060355040A131554484520555345525452555354204E4554574F524B312E302C06035504031325555345525452555354205253412043455254494649434154494F4E20415554484F52495459  "0\201\2101\0130\011\006\003U\004\006\023\002US1\0230\021\006\003U\004\010\023\012NEW JERSEY1\0240\022\006\003U\004\007\023\013JERSEY CITY1\0360\034\006\003U\004\012\023\025THE USERTRUST NETWORK1.0,\006\003U\004\003\023%USERTRUST RSA CERTIFICATION AUTHORITY"

There’s other commands to run too, see that comment

I was asking about potential “cleaner” or “security” tools, because I suspect some might have changed trust settings on certificates, disabled some, or installed some others, and that would explain why the verification fails (for Go applications, but not for, say, Safari).

I hope you find some more information based on that!

Thanks for the detailed update.

I’m officially out of ideas, seeing as: everything works for me on that same macOS version, and: someone who used to have the x509 error with previous butler versions no longer has it since the update.

My only idea would be to find the corresponding Root certificate and see if the trust settings are set to “Always trust” or “Custom”, see https://github.com/golang/go/issues/24652#issuecomment-378340252

I’m not sure you and Nikki Nyx are having the same problem.

I’ve read your messages too, but I can’t figure out exactly what problem you’re having. A screenshot would help!

When you get some time, could you try rebooting, starting the itch app, and trying to log in again?

I’ve gotten some more confirmations that the update did indeed fix that particular problem (the “x509” message), so I’m more and more surprised it’s not fixed for you.

A last minute theory could be that even though the files were deleted on disk, the older version was still running, and the installer “brought it to front” instead of starting a fresh copy of the app. (*nix systems like macOS don’t have problems with executable files being removed while they’re running, unlike on Windows)

(4 edits)

Alright, so, the setup works, itch starts up, it installs the latest version of butler (you don’t have to “choose it from preferences”, it’s automatically used, since you have it on disk it’s using it), and you’re seeing the exact same error on login, ie. this one?

Post https://api.itch.io/login: x509: certificate signed by unknown authority.

Edit: also, what’s your exact version of macOS? I’ve seen multiple reports of this error but haven’t been able to reproduce it across multiple machines and asking specific folks to test it :/

Edit 2: I see you mentioned El Capitan earlier (my bad), unfortunately that’s exactly what one of the versions I tested earlier today.

Are you running any specific software that would “enhance the security” of your mac? I see you’re using AppCleaner, anything else?

Edit 3: If you’re comfortable with the terminal, you can try to download https://broth.itch.ovh/butler/darwin-amd64-head/45dab99ef1c1908cc18b3b86bcdb0dad78de7789/archive/default then extract it, run “chmod +x butler” if needed, and run “./butler diag –all”

Edit 4: by the way, my current theory is that the issue is linked to that Go issue - it seemed to result in a fix, hence my trying to rebuild butler with a newer Go version. There are some troubleshooting steps in there that might help you. If you find a solution, please post it here!

(2 edits)

Hey Nikki,

Any particular reason why you launched the Setup instead of the app directly?

Also, was the app running when you ran the setup? This might explain why it gets stuck at “moving it into place” - macOS might not like it trying to replace a running application.

I’m going to try and reproduce that particular problem, then fix it, but in the meantime, you can quit the setup with Cmd+Q and just run itch by doing Cmd+Space, searching for itch and pressing enter.


Edit: I’m not able to reproduce the issue, running itch Setup while itch is running works just fine for me:

Can you quit the app (Cmd+Q) and try again now?

I’ve just deployed a new version of butler just for this macOS bug, it should show up as “butler 15.7.1” in preferences:

No, an API key problem would show as “invalid API key”.

Is there anything significantly different in that computers connection to the internet, compared to your previous setup?

Not being able to make the request in 2 seconds seems strange. I could try bumping the timeout to 5 or 10 seconds?

Although there’s already a “connect” and “idle” timeout in httpkit, maybe that one isn’t needed.

I immediately get the following response:

How quickly exactly? Would it be approximately two seconds?

Because that’s the timeout for API requests for this command:

https://github.com/itchio/butler/blob/d0b6d3e011414c67d2e16ee88968befe20700905/mansion/context.go#L119-L122

The app should be compatible with Windows 7+ and above.

Could your customer contact https://itch.io/support directly, with a screenshot?

Thanks.

Hey there,

I've identified a few bugs today, and this is one of them - you don't have to do anything, I'll release a fix in the coming week that will completely solve that issue.

I would still recommend putting save data somewhere else though - by putting it in the game folder, it'll get overwritten when the game updates (with the correct version, once this bug is fixed..), which may be bad for your players that want to hang on to their saves!

I've implemented a check for this particular error in butler, I'll deploy it soon:

(1 edit)

Hi, the issue is that the first zip you pushed was broken, but in a way that I'm seeing for the first time and that butler does not check for.

In particular, it contains both a file and a directory for some paths, like `MonoBleedingEdge` and `MonoBleedingEdge/etc`:


This, of course, makes no sense, and causes problems for butler's backend, but also, Windows' built-in compressed folders tool:


And 7-zip:


(Just to show that it's not *just* butler's backend and it really is a broken zip).


To resolve the problem, I'd like you to:

  • Tell me what tool you used to make "Eat Grow Evolve.zip" in the first place, so that my curiosity is satisfied and I can tell others to stay far away from it
  • Push the "Eat Grow Evolve Standalone" folder directly, without making a .zip first

Pushing a .zip with butler is allowed for convenience, but it saves no time at all - in fact, it wastes time, because butler has to decompress the .zip while uploading. (Compression is done once while uploading, and once again in the backend - so the compression done locally to create the zip file is always thrown away. This is necessary because butler needs to work on uncompressed files for diffing/patching to work well).

Edit: as for the current upload, you may as well delete it. Since the canonical .zip file for the previous build is invalid, no matter how you push to that channel, it'll fail to apply. It will also fail to install/extract for anyone in its current form. I'll add a check to try and detect this particular kind of problem when pushing .zip files.

Hey nick^4,

I tried to find the upload to show you how to use 'butler validate' to check your manifest, but I couldn't - the two most recent Windows uploads I can find for your account don't have an app manifest at all.

Anyway, as described on https://itch.io/docs/itch/integrating/manifest/validating-your-manifest.html - you can use butler to validate manifests (and entire builds). Ran on your two of your uploads, they give this:


My guess (unfortunately without being able to confirm since I don't know which upload you're talking about) is that you put a manifest in `standalone/.itch.toml`, whereas it should just be in `.itch.toml`.

The problem is that there is a top-level folder in your .zip file. If you were to use butler to push your uploads, you could do `butler push standalone nicknicknicknick/someproject:windows` (where `standalone` is a folder), and it would do the right thing.

TL;DR:

  • Manifests need to be at the top-level
  • Running `butler validate` then `butler push` is the recommended, safe way of doing things

Still not on our roadmap. The official installer from https://itch.io/app should happily run on most distributions. If it doesn't, open an issue!

The current distro packages available for itch all rely on itch-setup, so there's no need to update all of them when a new version is out.

When i tried to update the 32 bit, it automatically overwrote the 64 build. Pretty sure they were name differently.

You need unique channel names if you want to go that way, for example `windows-32` and `windows-64` (file names do not matter).

The itch docs https://itch.io/docs/itch/ are a good resource if you're not sure what to do.

In particular, for Windows https://itch.io/docs/itch/integrating/platforms/windows.html it says: "When in doubt, ship a 32-bit build".

I don't recommend bothering with a 64-bit Windows build, unless you have a very good reason to.

bonus question, since it overwrote and i had to reupload the 64 bit, it now says its 2 versions ahead can i edit that?

Yes! By default the "version" it shows is just an increasing number, per-channel. However, you can specify a version number, see: https://itch.io/docs/butler/pushing.html#specifying-your-own-version-number

Hey there,

I maintain butler & the app. The issue here is that you *just* switched over from web uploads to butler uploads. There's no way for the app to tell what it should upgrade to.

Anyone who installs now (or upgrades to 1.3) will get all the updates automatically.

After some looking around, I've unlocked external keys for both these users on those purchases.

The best way to get this fixed is to have the buyers themselves get in touch with https://itch.io/support - that way we don't have to hunt for the purchases.

(1 edit)

Hi, external keys should now be available for all your Winter Wolves purchases.

This part of the plan has been executed:

  • Iron out the last few corner cases with the current feature-set
  • Release app version with experimental "extract while downloading" feature

The rest, not yet. I'm considering using XFLATE instead of a separate blocks-based format. There were other priorities in the meantime though.

Note that (if bandwidth is not an issue) you can easily do the steps manually using `butler fetch` and `butler push`, to copy from one channel to another. 

There's a space in your path. Also, the target is `user/game`, not an URL.

The correct command is:

butler push "C:\Users\Kai\Documents\Unreal Projects\Who_could_that_be" ink-ribbon/who-could-that-be