The Hype vs. The Reality
Everywhere you look, people are debating AGI timelines. Some say it's two years away, others say twenty. After spending countless hours trying to build genuinely useful AI tools, I've become more skeptical of the near-term predictions. The models we have today are incredible, but trying to integrate them into real-world workflows reveals fundamental limitations that aren't just minor hurdles. They are roadblocks.
Many argue that even if AI progress stopped today, current models would transform the economy more than the internet. I disagree. The reason large companies aren't completely rebuilding their operations around LLMs isn't because they're slow to adapt. It's because getting reliable, human-like work out of these models is genuinely hard.
AI Doesn't Actually Learn
The single biggest issue is that LLMs don't improve with experience the way a person does. The lack of continual learning is a massive bottleneck. I've tried to build LLM agents for simple, repetitive tasks: cleaning up customer feedback from different sources, drafting release notes, or co-writing product requirement documents. These are language-in, language-out tasks that should be a perfect fit. And the models are maybe 7/10 at them. Impressive, but not transformative.
The problem is you're stuck with the out-of-the-box capability. You can endlessly tweak the system prompt, but you can't give the model high-level feedback that leads to genuine improvement. A human's real value isn't just their baseline intelligence; it's their ability to build context, learn from mistakes, and discover small efficiencies over time. AI can't do this.
Onboarding Analogy
Think about how you onboard a new junior software engineer. You don't just point them to the entire codebase documentation and assign them a critical bug fix. That would be a recipe for disaster.
Instead, you start them with a small, well-defined task. They submit a pull request. A senior engineer reviews it, not just for correctness, but for style, adherence to unwritten conventions, and efficiency. Through this interactive feedback loop of code reviews and pair programming, the junior engineer gradually learns the team's specific "way of doing things." They build a mental model of the system's architecture and the team's philosophy.
This would never work with today's AI. No amount of documentation fed into a prompt can replace that dynamic, iterative learning process. Yet this is the only way we can "teach" an LLM. It can read the manual, but it can't truly become a contributing member of the team.
This is why in-context learning, while useful, is a temporary fix. An LLM might grasp my product's tone of voice by the fourth paragraph of a session, but that understanding vanishes the moment the session ends. Compacting that rich, tacit knowledge into a text summary is a brittle and lossy process.