Progress Update on v3

This project is a struggle.

I started from a mistaken premise, "v2.0 played z3 and z4 games start to finish perfectly. I'll just add the z5 missing features, debug those, and I should be in good shape." My goodness, how wrong I was.

Current status

z5 and z8 games load and "play." Some games work better than others, and some will crash out almost immediately. Every game exhibiting new, exciting bugs! In researching the new bugs, I discovered long-standing bugs in Status Line v2.0 that, quite frankly, mistify me that the original even worked at all. So I'm rewriting parts of the code that control deep-level memory lookups and the project's stability rises and falls dramatically throughout the dev cycle. Such is the struggle. And the fun! (but don't discount the struggle)

Debugging on the Pico-8 is a drag

Pico-8 enforces a plethora of artificial limitations on the programmer, to evoke a feeling of programming in the days of old. As the platform has matured, developers have pushed Pico-8 further and further. Unfortunately, there has never been a great development cycle for more advanced projects. Missing dev features could, theoretically, be shoehorned into a project's codebase to provide a better dev experience. However, the developer then runs into the codebase size limitations of the platform, which prevents expansion of the project's features. You can have the handy dev code or you can have space to write your game; it's almost impossible to have both simultaneously.

For example, I've had to remove (during dev process) the save/load routines so I can have space to fit the debugging code and meet Pico-8's 8192 token limit. Comments have to be rewritten or removed to meet Pico-8's 65k character limit, but now I'm missing the comments that help explain things to myself. A tool that could automate tests simply doesn't fit into the space I have available. This all means that the project is rarely in a state where everything is working and available; subsystems that seem suspicious have to be isolated and repaired independently of the full, working project.

To gain insight into where data is being stored, and to verify that I'm reaching the correct bytes during operations, I need to add code for dumping tables of data. But there's literally no room in the codebase to add this, without disabling all of the other infrastructure that helps me debug other things. So there is this constant game of removing/re-establishing debugging code so I can investigate something else.

And while I'm at it, my KINGDOM to be able to breakpoint and step-in/over, and view Pico-8 objects in memory with ease!

It is a slow grind, is what I'm saying.

z-machine documentation is confounding

I've heard devs remark at the "clarity" of the existing documentation, and I just don't see it at all. Let's take this example about how "properties" are stored on game objects, specifically about the "size" of a property (how many bytes it takes to define it).

z-spec11 says, "the size byte is arranged as 32 times the number of data bytes minus one, plus the property number" which is true, but the effects of that are not immediately clear. It's frustrating, extra mental juggling, just to realize the top 3-bits hold the property number.

how to emulate the z-machine says, "A property entry contains one (one or two) size byte(s), followed by 1 to 8 (63) bytes of property data; the (first) size byte holds the number of data bytes minus 1 in the top 3 (2) bits" which clarifies things! But then it also adds that extra layer of confusion with the (#) to mix z3 and z4+ differences into a single sentence. Words are plentiful! We can use as many as we need! We don't have to squish multiple thoughts into one sentence!

Then we read on and find that v4+ has an exception, "if the top bit of the first size byte is %1, i.e., the number of bytes would be either 3 or 4, there are two size bytes." This means that the size could be much, much larger than 2 bytes for v4+, which we are then later told that if the size is over 2 bytes "behaviour is undefined" for certain calls. So now the exception has it's own exception. Hats on hats!

Then we read further into how to emulate and learn that this value (the length in bytes of the property) is stored, "V1-3: the top 3 bits; V4+, top bit is %0: bit 6; V4+, top bit is %1: the bottom 7 bits." and that's just flat-out wrong. In fact it turned out Status Line v2.0 had implemented in-part the WRONG definition and somehow it was working just fine? Right now, in debugging property-related things, the WRONG implementation is working better than the correct one. I don't get it.

Thank goodness for open-source projects that work properly to learn from!

(and I JUST NOW found this online document by someone with the same struggles, trying to clarify matters; looks good!)

Observations

• z8 games are slow to respond to user input. Not unbearable slow, I think? But definitely much, much slower than z3/4 games were. Log files balloon quickly when playtesting z8 games, generating files beyond the Pico-8 limit of 10MB (yet another fun limitation of the platform). The system simply seems to be asked to do a lot of extra work. But this was to be expected and is not unique to Pico-8 or my implementation. Older systems have the same issue with modern games.
• z5 games feel relatively snappy to me, all things considered. Border Zone feels quite playable.
• There are any number of modern classics that simply cannot format well for the Pico-8's tiny screen. Spider and Web loads and plays, but location names in the status bar are far too long and text overlaps. Photopia is artistic in its text presentation and draws things off-screen a lot. Status Line will not really be a great platform for playing these classics for the first time.
• Timer-based game support will likely need to wait until a v3.1 or v3.2 update
• I have an idea for sound effect support in Sherlock and Lurking Horror, but that might wait until a dot update as well.