AI Agents and "Intelligence"

There’s something fascinating about giving an AI agent agency and watching it behave in ways that look intelligent. AI agents these days are doing it all: painting, planning, programming. Under the hood, it’s just linear algebra. Not “real intelligence,” right?

The more time I’ve spent working with and building these systems, the more I keep coming back to the question of what intelligence actually is.

Maybe intelligence is the ability to produce intellectual output: writing, coding, solving problems, generating ideas that look meaningful to us. If this is the definition, then these systems already clearly qualify.

But maybe that’s too shallow.

Maybe intelligence isn’t just output. Maybe it requires understanding.

I keep coming back to something Ilya Sutskever has talked about in interviews around next-token prediction and whether it connects to understanding.

Imagine a very complicated detective novel. There are many characters, many events, hidden motives, and things happening across hundreds of pages that all build toward a final reveal. And then on the last page, there is a sentence like: “the criminal is: ___”.

Now suppose your task is to predict that final word.

To do that well, you can’t just look at the last sentence. You need to have tracked the entire story: who did what, who had what motive, what information each character had at each point, and how events connect over time. You basically need to build some internal model of the plot in order to make a correct prediction.

So maybe understanding isn’t something separate from prediction at all, but rather something that emerges when prediction becomes good enough.

Merriam-Webster defines intelligence as “the ability to learn or understand things or to deal with new or difficult situations.” Under this definition too, these systems start to look surprisingly capable. With few-shot prompting or in-context learning, these AI agents can often handle new situations fairly well, even without explicit retraining.

And still, I hesitate to call these systems truly “intelligent.”

Maybe intelligence also requires something closer to lived experience, or at least a different kind of connection to the world. With these systems, you can see the mechanism. It’s optimization, weights, gradients. There’s no hidden layer of meaning you can point to.

But this leads to yet another question:

If human intelligence were eventually reduced to similarly mechanistic processes, would we stop calling it intelligence too? Or would we just adjust what we mean by intelligence? If an alien intelligence looked at us and discovered that human cognition could also be described in those terms, would it hesitate to call us intelligent as well?

I don’t have an answer. I just keep coming back to this question the more I work with these systems and the more they continue to improve.