Skip to main content

Indie game storeFree gamesFun gamesHorror games
Game developmentAssetsComics
SalesBundles
Jobs
TagsGame Engines

Would smaller content-addressed updates be useful for itch.io developers?

A topic by Orelvis created 5 days ago Views: 1,241 Replies: 9
Viewing posts 1 to 4

Hi everyone,

I’m working on an open-source project called CAVS. It is a content-addressed delivery layer for game updates and binary builds.

The idea is simple: instead of making players download the whole new build again, CAVS reconstructs the original artifact byte-for-byte while downloading only the chunks missing from the next version.

It does not replace the game format or engine. A project can still use normal files like Godot PCKs, ZIPs, asset bundles, or binary builds.

Measured on real Godot PCK exports over HTTP:

  • tps-demo: 247.60 MiB full download → 1.64 MiB update
  • Marble: 6.55 MiB → 0.14 MiB
  • GDQuest demo: 27.61 MiB → 8.70 MiB
  • Same-version re-fetch: 0 bytes
  • Reconstruction: byte-identical

I’m not asking for an official itch.io integration right now. I’m mainly trying to understand whether this could be useful for itch.io developers who ship frequent builds, large updates, DLC, or custom launcher workflows.

Repo: https://github.com/orelvis15/cavs

Landing: https://orelvis15.github.io/cavs

Article: https://medium.com/@orelvis15/why-game-updates-are-bigger-than-they-should-be-and-how-cavs-fixes-it-f9d8ccb3d965

Would this kind of tool be useful in your release workflow?

Admin

Have you compared it to itch.io’s own patch generation algorithm? https://itch.io/docs/wharf/

Not yet directly against Wharf, and that is a fair comparison to add.

So far I benchmarked CAVS against full zstd/zip downloads, rsync wire transfer, rdiff, xdelta3 and bsdiff on real game builds. But since Wharf is itch.io’s own rsync-style patching protocol, it should definitely be included as a dedicated baseline.

I would not claim CAVS is better than Wharf without measuring it. My current view is that they solve related but slightly different problems: Wharf is a patching system for old→new transitions, while CAVS is a content-addressed delivery/cache layer focused on version streams, persistent cache reuse, resumable downloads, cache repair and CDN-friendly storage.

I’ll add a Wharf comparison to the benchmark plan. That would be the right way to evaluate it for itch.io workflows.

Admin

Wharf does two patching phases btw, there’s the initial rsync-style patch generated locally by butler for the person uploading. Then it does a second server side pass with bsdiff to generated optimized patch for players. You can run all these stages locally with butler, more info here: https://itch.io/docs/butler/offline.html

Keep me updated, curious to see how your project turns out.

Thanks, that’s really helpful. I had been studying some of the ideas behind Wharf already, especially the rsync-style phase, signatures, block ranges, and how the old version can be used as a source during reconstruction.

I wasn’t fully considering the offline butler flow and the optimized player-side patch generation, so I’m going to dig into that part next and run a proper comparison against the full pipeline, not just a Wharf-style baseline.

I’m also interested in seeing which ideas could fit naturally into CAVS without copying Wharf’s model directly, since I really like the design direction of Wharf. I’ll keep you updated as I test it more. Thanks again for the pointer.

Thanks again for the clarification — it helped a lot.

I updated CAVS after your comment. It now includes real `butler offline` comparisons using `butler diff/apply/verify`, plus SteamPipe-style analysis, pack layout diagnostics, route comparisons, Godot PCK certification, and a new `cavs certify` command that generates a full update report.

The project is still local-only — not a CDN or hosted service. The goal is to help developers test and understand game update behavior before shipping.

Website and guide:

https://orelvis15.github.io/cavs/

I’d really appreciate help testing it with real game builds. I don’t have enough varied game content to cover all cases, especially larger projects, different pack layouts, Godot PCKs, folders, DLC-like structures, or frequent content changes.

In your benchmarks, you acknowledge that pairwise diffs can produce smaller downloads, then dismiss them because you need N² of diffs. That’s technically true if you want to go from one arbitrary version to any other arbitrary version in one step, but nobody actually uses pairwise diffs that way.

The most common use of pairwise diffs only creates diffs between adjacent pairs, because by far the most common use case is going from one version to the next version. This provides optimal download sizes so long as no versions are skipped. Only N diffs are needed.

If skipping versions is common, one neat trick is to only provide power-of-two diffs between versions that are themselves powers of 2, i.e. pairwise diffs between adjacent diffs in the sequences 0-1-2-3-4-…, 0-2-4-6-8-10…, 0-4-8-16-32…, 0-8-16-32-64… and so on. This only requires 2N diffs and allows you to move between arbitrary versions with only O(2 log M) steps, where M is the distance between the versions.

Finally, if new versions are very frequent and very similar to each other, there’s always the trick of setting one version (the first one, or the one after the last major change) as the “base” version and only storing diffs to and from the base version. This only requires N diffs (recalculated each time the base version changes) and 2 diffs to change from one arbitrary version to another arbitrary version, at the cost of potentially very large diffs.

That’s a really good point, and I think you’re right.

My N² argument only applies to the “all pairs, arbitrary version jump in one step” case, which is not how pairwise patch systems are usually deployed in practice. Adjacent diffs, power-of-two/sparse ladders, and base-version strategies are much more realistic baselines.

I’m going to update the wording and benchmarks to reflect that. Instead of framing pairwise diffs as “bad because N²”, I’ll compare CAVS against several practical patch policies:

  • adjacent-only diffs;
  • power-of-two ladder diffs;
  • base-version diffs;
  • all-pairs as the theoretical one-step baseline;
  • CAVS content-addressed / hybrid reconstruction.

I still think CAVS has an interesting tradeoff because it can reuse cache, previous installed artifacts, and content-addressed chunks without requiring a separate patch graph for every route. But you’re right that the comparison should be against realistic patch policies, not just the worst-case all-pairs model.

Thanks for pointing this out. This is exactly the kind of feedback I was hoping to get.

You were right — my previous wording was too focused on the all-pairs N² case, which is not the way pairwise patch systems are usually deployed.

I updated the benchmark to compare CAVS against more practical patch policies, including adjacent-only updates, skip-heavy scenarios, warm-cache cases, and replayable patch graphs. The new results are now documented in docs/results/v1.1.0/patch-policy/ and summarized in BENCHMARKS.md.

The goal is no longer to frame pairwise diffs as “bad because N²”, but to compare CAVS against realistic policies and show the tradeoffs in storage, update size, skipped-version behavior, apply path length, and cache reuse.

Thanks again for pointing this out — it made the benchmark much better.

Thanks again for the feedback. I tried to make testing the Godot flow simpler.

I created CAVS Desktop to make it easier to try the basic workflow without starting from several CLI commands. It is mainly meant to help with testing: select an old .pck and a new .pck, analyze the update, generate the CAVS data, start a local test server, and copy a minimal Godot snippet.

Download: https://github.com/orelvis15/cavs/releases?q=desktop

If anyone tries it with a small Godot project, I’d really appreciate feedback on whether this makes the flow easier to test.