Choral intelligences

The evolution from agentic AI to persistent, volitional systems

Around two years ago, almost to the day, I spent an absolutely frantic evening in Berkeley, going through enough coffee to power a mid-sized city, hammering away at my laptop on trying to figure out what comes after LLMs. What to most people was still barely on the horizon at the time has been a subject I have been working on in various capacities for the best part of the past decade, on and off. Transformers have revitalised the interest in AI for language that has for a while been eclipsed by computer vision, which in the preceding decade or so has gotten a significant boost from convolutional neural networks. But then Vaswani et al. came along with the Attention paper,¹, and suddenly language was cool again.

¹ Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, Vol. 30. Curran Associates, Inc., 5998–6008.

What I was trying to understand that night in Berkeley was how we could let these language models do more than speak. There’s a kind of magic to computer science, in that the barrier between the conceptual and the real, the thought and the action, is rather more permeable than elsewhere. In few other fields, if any, do we traffic with such frequency between words on the screen and the motion of a remotely controlled robot arm, a $10m stock trade or a drone strike – and vice versa. And so, it stood to reason that if LLMs could produce words, they ought to be able to produce actions, too.

I wasn’t – despite occasional assertions by many – the first to conceptualise ‘agentic’ AI. Agency has been a topic in AI for a long time. Russell and Norvig famously gave a definition of it in their seminal textbook,² and I keep showing that on a stark blue slide every time I give a talk on the subject to remind us of the giants’ shoulders we are privileged to stand on. Around the same time, at least two others have considered the same ideas – Harrison Chase, at LangGraph, and Lilian Weng, then at OpenAI, now at Thinking Machines.³ and arrived at the same terminology.

² Russell, Stuart J., and Peter Norvig. 2021. Artificial Intelligence: A Modern Approach (4th ed.). Pearson. https://aima.cs.berkeley.edu/

³ I’d like to say that great minds think alike, but that would make me look awfully out of place between the other two.

In retrospect, there are moments I regret the terminology I ended up adopting in my post on the subject. I’m not sure it hasn’t, inadvertently, led to human harm. What I did not necessarily think of at the time was that I came to the word from a very specific angle. I’m of course a recovering lawyer, where agency is a very well-defined concept, and raises issues that those who primarily associate the term with airport ticketing or James Bond don’t necessarily think of (such as delegation and delegability). But more than that, I was thinking of the etymological roots of the word, from the Latin agere, from which our English word ‘action’ also derives – agentic AI was AI with the key differentia of being able to act, even if it is through some form of speech.

Pentheus and the Maenads

Classical Greek drama is incredibly diverse in its subjects, characters and tone. Yet one thing unites almost every extant play that we have in any non-trivial volume: there’s a bunch of characters, a dozen to fifty or so, who act as the chorus. They’re not actors (agonistes), but are just as important, if not more so. They comment on the action, they provide context, they provide a moral compass, they provide a collective voice. They address, and are addressed.⁴

⁴ Athena’s speech at the very end of the Eumenides is one of my favourites. In it, she warns her own people about the hubris of judgment without piety and the importance of fair government, neither tyranny nor lawlessness.

The term ‘chorus’ comes from the Greek χορός (khorós), which in turn derives from the Proto-Indo-European root gher-, meaning ‘to grasp, enclose’. The chorus is, in a sense, the collective that encompasses the action of the play. It is not an actor, but it is not a mere observer either. Nor has it the fleeting on-stage, off-stage nature of the actors. They are persistent and constant. Their reflections and witness spans the entirety of the play. They have, to use the LLM term, a long context window.

I’ve been thinking about this because we’re about to see something similar emerge in the AI systems we’re building. The agentic revolution or turn that I discussed in my past writings is, as far as the theory is concerned, more or less played out. What no more than a handful of us have thought of in the waning days of 2023 is now – for better or worse – plastered on billboards along the US-101 off Redwood City and a household word for everyone looking for a Series A, a raise or some Youtube views.

Since I’m not really after either of these, I have mostly been thinking about what comes next. There have been warning signs (like the recent failures of leading LLMs on the MCPMark benchmark) that suggest the limitations of transformers are becoming evident, and the growing scramble for compute does not exactly suggest an efficient scaling regime. Agents, don’t get me wrong, are great, and I expect them to be the dominant paradigm, or part thereof, for a good while.

But evolution doesn’t wait.

Agents were an evolution from chatbots and simple LLMs. The key improvement was self-directed action as part of a goal behaviour. In short, the evolution here is of increasingly volitional systems.⁵ Agentic AI was able to act in ways that need to fit into a triggering framework, but they can determine the parameters of that action, and in sufficiently large numbers, networked agents can display incredible complexity. Agents can act. Choral intelligences can will.

⁵ I find calling them more ‘autonomous’ to be a misnomer. Autonomy and volition do not have to coexist.

I need to be careful here because ‘will’ carries enormous philosophical baggage. I’m not talking about human motivation, consciousness or desire. I’m talking about something more fundamental and perhaps more interesting: the kind of will that’s implicit in environmental response. A thermostat wills the temperature to be 20 degrees in the sense that it continuously acts to make that state true. A coral reef wills its own persistence through the distributed actions of thousands of organisms, none of which individually possesses anything like intention.

Choral intelligences will in this environmental sense. They maintain a continuous relationship with world state, constantly curating their understanding, perpetually acting to align reality with their implicit goals. Not because they want to in any psychological sense, but because that’s what they are – ambient frameworks that surround and engage with their domains.

This is a profound shift from current AI architectures. Even the most sophisticated agentic systems today are fundamentally episodic. They wake up, accomplish a task, and go dormant. They respond or act, but they don’t continuously will. A choral intelligence, by contrast, is like the Greek chorus – always there, always processing, always maintaining its understanding of world state.

Into the world

Attention gave us agents by providing for a way to maintain temporal consistency and contextual relevance across sequences. Before attention, we had models that could process text but couldn’t really understand how different parts related to each other across distance. Attention gave us that, and from attention came the transformer architecture, and from transformers came everything we now think of as modern AI.

But attention’s children are reaching the limits of what they can do on their own. The transformer architecture, for all its elegance and power, has a fundamental constraint: it processes in discrete steps, with finite context windows, and it doesn’t maintain genuine world state. It can reason about the world, but it doesn’t model the world as an evolving, persistent thing.

This is where world models come in, and why they’re not just an incremental improvement but a necessary foundation for choral intelligences.5 World models are attempts to build representations of how things actually work – not just statistical patterns in text, but actual models of causality, persistence, state change over time. They’re the kind of long-term ‘theory of mind’ reflective abstraction that a choral intelligence needs to function.

When I watch the current generation of world model research – the work on physics simulators, on learned dynamics, on models that can predict how scenes evolve – I see the necessary infrastructure for systems that can will in the environmental sense I described. You can’t maintain a continuous relationship with world state if you don’t have a model of what world state means, how it changes, what persistence even is beyond token sequences.

This gives us what I consider the three crucial features of choral intelligence:

• persistence
• state
• spontaneous, volitional reactivity

A choral intelligence does not need to be prompted. It is ubiquitously present, and carries a world state representation that it updates and adapts. In this, it reflects a fundamental human concern. Imagine you woke up in a different city every day, without much of an ability to predict where you’ll be next.⁶ We expect the things we rely on to exhibit consistency, and associate few good things with unpredictability in general. 1m token context windows are nice, but a long shout from satisfying this deep human need for persistence. Because of the way transformers work, context windows scale computational cost on \(O(n^2)\). The persistence humans need AI to exhibit is not on the table with current designs. The chorus is our way out of this predicament.

⁶ It’s the time of the year when that is actually eerily close to my lived experience.

flowchart TD
    Chorus["Chorus"]
    WM["World model"]
    Time["Time"]
    User["User"]
    State["State"]

    Chorus -->|Spontaneous,<br/>volitional reactivity| WM
    WM -->|Environmental<br/>feedback| Chorus

    Chorus -->|Persistence| Time
    Time -->|Continuity| Chorus

    Chorus --> State --> User
    User --> State --> Chorus

    State -.->|Temporal coherence| Time

    style Chorus fill:#4fc3f7
    style WM fill:#b3e5fc
    style Time fill:#b3e5fc
    style User fill:#b3e5fc
    style State fill:#e1f5ff

The chorus and its fundamental interactions

Forming the chorus

So, how do we build one? The short answer is that we don’t yet know. But we do have some clues. Most of these derive from what I call ‘transformer-plus agents’: agentic systems that are firmly reliant on transformer-based LLMs, but use agentic frameworks to shore up their shortcomings. A classical example is, of course, memory augmentation through an extraneous state memory store to extend context. This is by definition incapable of creating actual persistence, but it does create a convincing enough simulacrum of it by extending the context window in practice. But this only solves half the problem – that of recording state. The other half is to do something useful with it. That is where world models come in, which at their crudest level can be described as encoded probability distributions of possible versus impossible world states given a set of past world states.

The final piece is the volitional-reactive element. To me, this necessitates a fundamental rethinking. Consider the Great Barrier Reef.⁷ It’s a vast, complex ecosystem made up of thousands of species, none of which is particularly sophisticated. Yet together, they form a system that collectively is capable of maintaining a sort of homeostasis (until humans enter the scene) that is beyond the capabilities of our supercomputers. We do not have the compute capacity to operate a control system that would be capable of managing what a bunch of invertebrates, algae and bacteria can pull off.

⁷ Coral comes from the Latin corallium, which in turn derives from the Greek κοράλλιον (korállion), which probably comes from the proto-Semitic gor meaning ‘stone pebble’. It’s not cognate with ‘choral’.

The model, then, for the chorus is not chasing ever larger context length monolithic models, or workarounds to simulate one. Rather, it uses the composability of small, simple and limited agents that each have a narrow domain of expertise and a specific function, and just enough knowledge to reflect a tiny facet of the world. They are, in a sense, the coral polyps of the AI world. We do not need them to hold or operate or conceive of a world model, nor do we need them to have million-token context windows. We might not even need elaborate inter-agent communication protocols: as phenomena like quorum sensing show, complex collective behaviour can emerge from very simple local rules and interactions. It’s the emergent capability of these systems that matters, not the sophistication of the individual components.

Nor is a choral intelligence necessarily sharply differentiated from conventional agentic AI. The same, by the way, is true for agentic AI itself. One of its greatest successes is that it is a statement of degree, not kind. There is – despite assertions to the contrary – a rather considerable penumbra of systems that may or may not be considered agentic. This is a feature, not a bug, and was rather inherent in my formulation of the concept when I first drew out an example of an agentic system. Just as there are deep neural networks and even deeper ones, there are less and more agentic systems, from a few coupled functionally defined modules to self-organising swarms. Agency is fuzzy around the edges. This has allowed us to develop and grow into it, rather than debate its definition ad nauseam. I expect, or at least hope, for the same for choral intelligences.

What is clearer is the transformational pattern. We are already seeing a recognition that this is the aspired evolutionary pattern. Just last week, OpenAI launched Pulse, which at the very least seeks to emulate the user experience of a choral intelligence⁸ by curating a newsfeed based on interactions and data sources like my calendar.⁹ Pulse is not, by any stretch of the imagination, a choral intelligence. But it does seek to be always present, always available, always responsive. This is a first step, and many more will follow as AI moves from prompted (chatbots) through programmed (agents) to present (choral).

⁸ I maintain – to the great chagrin of my friends who work there – that OpenAI is an okay AI company, but the best UI company in the world bar none. Their biggest hits aren’t necessarily cutting edge AI, but creating a way to make it accessible and usable. ChatGPT is a great example of AI < UI: OpenAI didn’t invent GPT, but they made it into something with just enough comfort and skeuomorphy to speak to the user ‘as a man would to a man’. They reached into the deepest human desires: companionship, understanding, being understood – and delivered on the emotional yield of it. That is great UX.

⁹ Or someone else’s. The other day, it gave me a great travel guide for Lisbon. I’m… not in Lisbon.

Intelligence under will

In thinking about what comes after the agentic turn, I keep returning to that image of the Greek chorus. Always present, always witnessing, always providing the framework within which everything else happens. Not the protagonists, not driving the action, but essential to making the action meaningful.

Perhaps that’s what we’re really building with choral intelligences: not artificial minds that replace human thinking, but ambient frameworks that surround and support human activity. I don’t see choral intelligences as replacements of human judgment – indeed, they are arguably more suited to augment humans than current models, for they seek to satisfy our deep human need for continuity –, nor do I see them as part of the current pursuit of AGI.¹⁰ I see them not as superintelligence but ambient intelligences (plural), not replacements but augmentation through continuous, coherent environmental willing.

¹⁰ I have never been able to get excited about AGI. I’m as much a messianic zealot about some aspects of AI as one can get, but I’m also a pragmatist. The embedding that corresponds to the vector between messianic zeal and healthy pragmatism maps very closely to ‘Kool-Aid’.

¹¹ The progression feels natural in retrospect, almost inevitable. But living through it, building it, trying to understand what it means – that’s still thrilling. I have been privileged to be afforded a small place in this great story, and if I could have one wish, it would be that whatever this ends up amounting to in the end, it would be something I can say was for the betterment of all.

Two years ago, a few researchers in rather obscure corners of the AI community independently came to the conclusion that if we equipped LLMs with tools and the right impulses, we can turn speech acts into pizza.¹¹ The agentic turn has mostly delivered on that, despite its shortcomings: I am, as I write, waiting for Instacart to deliver groceries ordered by a multi-agent system I built that uses a highly customised Llama 3.1 70b instruct model for its core reasoning functions. What comes next, however, will transform not what AI can do, but how, when and why it does it. From reactive systems to perceivers that are persistently evaluating a world model generated through a distributed network of simple atomic intelligences, we will be moving into a transformation that changes not just the capabilities of AI, but its very character.

The chorus encircles, surrounds, provides the ambient framework. It speaks with many voices that somehow become one voice. It witnesses everything whilst also shaping through its responses. And it’s always there, maintaining continuity across scenes, remembering what has passed, anticipating what will come.

The chorus never exits. From prologue to exodus, they remain – shaping through presence, witnessing through persistence, willing through continuous engagement with the world. That is what we are building. That is what comes beyond agents. The age of ambient intelligence begins not with machines that respond or act, but with machines that abide.