@redonihunter - So forgive me, but also correct me, on my jargon here. My entry point into higher mathematics was probably the most round-about way possible. I am still adapting my vocabulary. What I speak to here, I am referring to a sense of "invariability".
Just from my own perspective, the term "training" implies something I have not yet witnessed. The weight calibrations feel like exactly that, calibrations. When I sit an think about it, I suppose I was harsh on this term. I think I may have sort of "flash-formed" direct connections to the overall epiphany because it all hit me at once. When I look at modern AI, all I can seem to see is something like this:
Vectors ~= Coordinates
Matrix ~= Map
Weighted Matrix ~= Re-drawing or re-calibrating the map
Weights ~= Calibration
Bias ~= Threshold or Condition
Knowledge Base ~= Packaged Guidance System
Temperature ~= How Far You Are Allowed To Arrive From An Intended Destination
Then you add in the softMax? That looks like some kind of Sci-Fi high-dimensional probability drive where every "neuron" is like a different landing point and every "synapse" is the jump through hyperspace. Then, at the final layer, whichever little spaceship warped closest to the desired token space wins. I mean, we're talking passing the mathematical expression of a thing through high-dimensional space, nudged along by little weighted bumps and then locked into better and better patterns of navigation. This isn't even pattern recognition, this is something else entirely... Like... It reminds me of how water can erode rock and carve channels. Almost like the frozen weights represent some sort of "tunneling" of some kind.
But anyways, it appears that nothing ever actually touches the network itself. Like, no one ever let it upload files and chew up color patterns to mimic stuff with common pattern recognition. None of that stuff. You know what I mean?
And by "there is no output" I know that's a bit of a stretch but literally speaking, the output is the input transformed by this process. Take a ChatGPT conversation for example. Every token (let's say word) activated is a lightning-fast automation of this mathematical process. Every word it prints, it has fed the entire conversation back in over and over. Including the words it, itself generated.
Talking to a chat AI is not a conversation. It is a composition. This is why context management is so important. It's why it "lies". It's why it has "memory" problems... There is no memory. There's nothing to remember.