itch.io Community » itch.io » itch app Development

does itch app actually resume downloads?

A topic by reddesertdisk created May 11, 2020 Views: 1,712 Replies: 6

Viewing posts 1 to 2

I found it hard to resume downloads via browser and since I'm on prepaid internet I didn't want to waste limited data.

I found a sort of workaround by waiting for the download to start on the browser, cancelling it, and copy-pasting it in IDM. Repeating it if the link expires.

But for large files (>1GB) I find it too risky since the downloads end up corrupted and I can't open the zip/rar files.

So I tried the app and was surprised to find it had the option to resume downloads.

I tried it on a 1GB game and had to pause it at 70% percent or so (in the progress bar), but once I resume it the progress bar goes back to the start at 0% without so much as a warning. I felt like I just wasted a bunch of data.

Can I please get an honest answer about how downloading via the app works? Does it behave like a torrent and download parts of a file and merge them? Or does it delete everything and start from scratch?

Like Reply

Amos4 years ago

Hey, I maintain the itch desktop app.

It does actually resume downloads. Here’s what it’s supposed to look like:

https://youtu.be/g3WyjRo4oj0

However, depending on the contents of the upload, the app has a few different strategies.

For formats like .zip, which is what you get if developers use our tool for uploading games, the situation is ideal. The app actually manages to extract files while downloading - so the .zip file is never actually stored on disk, which saves on disk I/O and disk space.

The tech involved is actually pretty neat - the app stores the state of the zip decompressor (I just call those “checkpoints”) on disk every few seconds, which means it can resume even if you quit the app and restart it, or if your computer is shut down.

For formats like .7z and .rar, “streaming decompression” is impractical due to how those formats are designed. The best we could do is save checkpoints after every entry in the archive. And some games, like UE4 games, typically ship one gigantic .pak file and then a few small files. If downloading was paused in the middle of extracting the big .pak file, then that progress would be lost.

So we don’t do that. For those formats, like .7z and .rar, we just download the entire archive file to disk first. And then we decompress from the disk, to the disk. Which means you see the progress bar go from 0 to 100 twice. Once when it’s downloading, and the second time when it’s extracting.

So - as long as everything is working as expected - the only time you should see a progress bar restart from zero when you resume it, is if it was 1) a .7z/.rar file 2) the file was fully downloaded to disk already 3) it was in the middle of extracting it.

Even then, we do save checkpoints while extracting from disk to disk - to avoid re-extracting the same files over again on resume. So as an additional condition, it would have to be an upload from an UE4 game, or another game that has is mostly one huge file.

So I guess my question is: what were you trying to install? I’ll definitely want to try and see if I can reproduce the problem locally.

Like Reply

reddesertdisk4 years ago (1 edit)

Thank you for the response.

The game is Type Dreams by Hofmeier (https://hofmeier.itch.io/type-dreams)

I've got a few more questions if you don't mind.

Am I right to assume that :

1) Even when the progress bar restarts from zero when resumed, it doesn't necessarily mean that it's starting from the beginning (or re-downloads everything from the beginning) and merely resumes from where it had stopped downloading/extracting.

2) When it still displays 'KB/s - minutes' it means that it's still downloading, not extracting.

3) When I checked 'AppData\Roaming\itch\apps\type-dreams' there were files in that folder. Does this mean that these were the files that were downloaded, and will not need to be re-downloaded by the app again?

Like Reply

Amos4 years ago

Type Dreams is distributed as a .zip file, so the “good” strategy is used - the .zip is never stored on disk, and instead extracted on the fly.

In that case, you’d see all the files on disk as soon as the download starts. They’re just preallocated at the start of the single download+extract phase. (That should answer 3)

For 2, you’d only see a second progress bar if the format was something other than .zip, so not the case here.

For 1, well, if you had a progress bar restart from zero while downloading Type Dreams, something else went wrong, and I might be interested in looking at your app log: you can find it in “Preferences” => “Advanced” => “Open app log”, from that window you can type “Open folder” to locate the file. Or you might be able to diagnose just from reading the messages :)

Like Reply

reddesertdisk4 years ago

I just pasted the part where I saw 'Resuming @ 57.7%' change to 'Resuming @ 8.4%' later on

https://pastebin.com/wVC9AHwu

Is this enough or should I post the whole log? and if so where should it be posted?

Thanks again! :)

Like Reply

Amos4 years ago

Thanks for the log. Still, it’s kinda hard to tell what’s going on.. I did notice a bunch of network errors when the app was doing its self-update check, and when downloading locales…

Do you have any third-party antivirus running?

Like Reply

reddesertdisk4 years ago

Here's the rest of the Itch log, I hope it's not that daunting to sift through.

I did turn off all antivirus apps when I downloaded through the app, even Windows Defender and Firewall, so that they couldn't share bandwidth.

I do have something turned on called IObit's 'Driver Booster,' though I'm not quite sure if this is the one causing trouble.

If there's really no solution I just might revert back to my browser-IDM plan and hope the zip doesn't come out corrupted...

Thanks anyway for taking the time for answering my questions! :)

Like Reply