Forgive me for the double reply, I read your message first then replied and am now watching the video you posted.
So far, immediately, there's a problem and maybe it's half-mine. This explanation is not what a "token" is. At least not in the system architecture sense. But now I am connecting the dots... Ok, so they call the limit a "token" limit because they charge you per token (navigation point) reached at the end of each full pass. Ok... Still watching...
Yep. A token isn't always a full word. Right. Still viewing...
Yeeeaaahhh dude! They were DECIDED and realizing that blew my mind. From my perspective, it's like writing a map of a place you've never been and then navigating probability until the landscape matches the map you drew... It's wild!
Boom--right there! Did you catch that at 4:04 to 4:10? "It doesn't care, they're just input tokens" That right there, the "it" he's talking about is the neural network. It doesn't care because it's a high-dimensional probability drive. The output is repurposed through the surrounding framework into "intelligence" and "behavior". This is what I am pointing to, the actual system itself before any arbitrary tokens are involved. The tokens are part of the framework, not the system itself... Still watching...
So, I don't want anyone to take this the wrong way... I'm not saying it's a conspiracy or anything... But the very next thing the guy does if you listen to his words and also pay attention to his body language: "But that's not even the interesting part" and he waves it away with a physical gesture. Before I taught myself linear algebra, I came from a 30+ year background of psychology and philosophy (b.s. in humanities) and that dude got uncomfortable when he realized where he was about to have to go to explain "why" the tokens don't matter. I don't think it's any kind of conspiracy, I think it might be a sense of embarrassment like the Tech world doesn't want to admit how far away from AGI modern AI architecture actually is.
I did watch the rest of the video but the rest of it revolves around token pricing... Okay... I see what's going on now. There is no real measurable way to price the usage of something like a modern AI because it's literally just automated calculations. So I can see the logic behind charging by the token.
Something isn't adding up, man... You don't need all these data centers and all this stuff to achieve something like ChatGPT. Even if everyone in the world was using it, yeah you'd need to scoop up all the RAM and processing power you could but you don't need all this storage space unless you're saving every single output of the neural network thought process and every single conversation.
I dunno... Anyways, that was a good video. I get it now, the token pricing versus actual tokens. Makes sense.