The Missing Piece of Artificial Intelligence
There is a glaring gap between human intelligence and the artificial intelligence we build today. Humans learn continuously. We make mistakes, update our beliefs, acquire new skills, and refine our behavior over time. We get better.
Most AI agents, by contrast, are stuck in a "Groundhog Day" loop. They might have access to incredible world knowledge, but they don't meaningfully improve from their experiences. They are static entities, frozen in time at the moment they were trained.
The traditional answer to this problem has been to focus on the neural network's "weights" the billions of numerical parameters that define the model. Researchers have spent decades trying to figure out how to update these weights on the fly without breaking the model (a problem known as "catastrophic forgetting"). But despite all this effort, modern production LLMs are still deployed with frozen weights.
We are looking for the solution in the wrong place. The breakthrough for creating agents that truly learn doesn't lie in the complex math of weight updates. It lies in a much more accessible and powerful concept: learning in token space.
Why Updating Weights Doesn't Work for Agents
The old way of thinking assumes that "learning" equals "changing the model." But this approach has proven to be incredibly difficult for practical, deployed agents.
- It's Too Risky: Training a model requires careful data curation and human oversight. Trying to automate this process for every single agent in real-time is a recipe for disaster. One bad update can ruin the model's performance on everything else.
- It's a Privacy Nightmare: If you have millions of users, whose data do you use to update the model? Mixing everyone's data into a shared model risks leaking private information. Training a separate model for every user is prohibitively expensive.
- The Feedback is Too Simple: Most weight-update methods rely on simple "thumbs up/thumbs down" rewards. This is a very low-resolution signal for complex tasks. Agents need rich, detailed feedback, not just a score.
The Power of Token-Space Learning
Modern AI agents are not just a model. They are a system composed of the model plus its context. This includes the system prompt (instructions), the tool definitions (capabilities), and the conversation history (memories).