CIE LAB, while the most popular perceptual color model, is super outdated and flawed. JzAzBz is the best color model that I know of, which extremely straight hue lines and uniform MacAdam ellipses. The coordinate system is very similar to CIE LAB so no one should be lost when using that instead of LAB, or alongside LAB if you would want LAB to remain despite its flaws.
Something to note about the article that tripped me up at first was that you are to make your own inverse matrices and I believe somewhere in the conversion from L'M'S' to XYZ it wanted you to use elements of the 3 coordinates that you are currently working on to solve the last 2 or 1 coordinates. Just to save a headache in case we have the same problems.
A really obvious glaring issue with CIE LAB that anyone with Photoshop can see is how it handles Blue. Set the vertical strip in Photoshop's color window to LAB's L and set the hue to Blue and you'll see that when it's dark it's Blue as you would expect but as it gets brighter it becomes Cyan. That's not perceptually the same hue, Blue and Cyan are different.
Some less obvious issues, or rather benefits JzAzBz has over CIE LAB, is its measurement of perceptual brightness which, when compared side by side, is much better. The research article I linked also has a few graphs illustrating differences in mapping between LAB and JzAzBz, which looks more evenly spaced. I believe this image shows the MacAdam Ellipses in CIE LAB, though it might be something else entirely and I hope that you have better luck finding this than I had for comparing with JzAzBz's MacAdam Ellipses graph.
This should also help with automatically picking unique colors from images.
--- --- ---
While I haven't noticed any actual issue with RGB blending, I just want to verify that it's blending with Linearized Gamma. I'm not at my own computer right now so I can't blend between Green and Red and see what it does to know for sure, it's just a concern that came to mind while typing out the earlier bit.
--- --- ---
There's 3 Hue-based color selection approaches, HSV HSL and HSY / HCY, but there's no LCH which is the hue / chroma transformation of LAB. Ideally if JzAzBz is implemented there would be a JzCzH.
--- --- ---
Median filter on imported images to remove noise / blemishes. Like I've got beautiful paintings I bring in and they have cracks and other issues, on top of digital noise at times. I'm prepared to run median filters in an external program and have multiple copies of the same image on my HDD for the program to use but I'd rather it all be done live so I don't have to prepare multiple variations of images.
--- --- ---
A hilbert curve would be useful as an additional method of choosing colors from an image. https://corte.si/posts/code/hilbert/swatches/index.html https://github.com/cortesi/scurve It arranges colors in a very approachable manner, much better than trying to locate important colors when the surrounding colors are influencing its appearance and making it look like a different color, plus it's easier to click on colors since they get grouped together. Perhaps it would be even better when arranging based on JzCzH or JzAzBz instead of RGB? It would also be nice to run a Median filter first so that noise or scratches or other blemishes don't appear.
--- --- ---
Global non-destructive color filters such as RGB curves (like non-destructive adjustment layers in Photoshop) could be useful for giving a palette a certain tone without having to start off with colors that fit that tone, like purple shadows and yellow highlights which seem popular, and this can be done with curves while simpler and more approachable colors like straight up black and white can be used. Also with an option to "bake down" the color filters to the grid and perhaps a mask / intensity / opacity that can be painted in or out with a tool like the Saturation / Desaturation pen tools.
Paired with this, the ability to detect a color filter / RGB curve / tone in an input image (after a median filter to avoid noise / scratches / etc), toggle a correction for this on and off, select colors from the image, and add the detected color filter / RGB curve as a global non-destructive filter. This way the user can better see what a color is supposed to be behind all the lighting effects, decide if they want to include it, then apply the color filter or modify it in some way. It's always easier to modify colors when an image is properly white balanced, reds are better distinguished from oranges and other similar colors are better distinguished and are easier to specifically modify. Otherwise lots of colors that are perceptually different could all technically be different shades of yellow. White balancing works best with linearized gamma RGB curves done in a very large RGB color space, like ACES, in my experience.
My ideas for detecting color filters / tone curves / etc would either be to have the user specify colors that should be 0 saturation (not just middle gray) with the color picker, and have the program construct an RGB curve what makes those colors have the lowest saturation possible. This should cover shadows, mids, and highlights. Perhaps a blatant black / white level selection as well. The other idea is based on histogram equalization.
--- --- ---
Maybe 3D LUT / ICC Profile support for things like CRT simulations? Maybe? I don't really care too much about this specifically but someone might find it cool to use pure sRGB red, green, and blue and then load a 3D LUT that makes it look like a Sony Trinitron's color primaries. I would find it cool as well but I don't really feel like it would be as useful as anything prior to this.