I've been working on data compression stuff for my new bump-dash action rpg, Blintzy's Shattered Dream

in order to get the most out of pico-8's limitations, I checked out data compression. I found the px9 algorithm made by the creator of pico-8 themselves. I thought if I implemented this, I'd be able to make a bigger game with more sprites and maps! 

above: the game cartridge, storing the compressed data where the sprites usually are

it turned out to have some implementation challenges, but I believe I've made something quite nice for my needs. the core of this challenge is that those compressed data chunks need to go somewhere in the cart too, and I need a way to know where in memory the chunk I need is at.

in order to streamline this, I made some decisions about how the memory would be mapped: the first two pages (or the top half) of my sprite sheet will be used by the whole game, leaving the second two sprite pages and the map sheet to be swapped around during play. but I still want to compress the first two sprite pages, so I'd need to make some special functionality to load that at game start.

after that, I need a way to know where all the data is living in memory. in order to do this, I stored 2-byte numbers between each chunk of memory. each of those 2-byte numbers contain the address of the next chunk of memory, sort of like a linked list. then during startup, I can build an array of memory addresses to let me easily find where each data chunk is. 

above: the studio cartridge, where I edit sprites/maps, manage data and export them into the game cartridge

in order to make this all work together nicely, I made a second companion cartridge as my environment to manage the data. I call it the "studio" cartridge, since here is where I'll be drawing the graphics and maps from now on. it won't be distributed with the game, it's just a handy tool to manage the data. the actual compressed data stays in the game, letting me keep the game in a state that resembles the form it'll be released in. then, the studio cartridge can keep the sprites and maps. when the studio cartridge boots up, it imports the compressed data from the game cartridge and lets me import and export it as I please. then when the game boots up, it can copy the compressed data from the sprite area to the extra cartridge space at 0x8000 and begin decompressing things into memory.

the final thing I had to write was functionality to re-adjust data chunk locations. because the compression algorithm doesn't always save things to the same size, I needed something to move the data around when other data chunks get larger or smaller. this was the longest part of the process since it required some precise math, but I finally got there! 

it was really rewarding to create this. I still have anxiety about the amount of stuff I'll be able to fit in the game compared to my ambitions for it and the generally large sprite sizes, but let's see what happens! I can always scope down along the way.