Agentic AI and the Next Intelligence Explosion

  • Authors: James Evans (Google Paradigms of Intelligence Team / U Chicago / Santa Fe Institute), Benjamin Bratton (Google PoI Team / Antikythera Berggruen Institute / UCSD), Blaise Agüera y Arcas (Google PoI Team / Antikythera)

  • Original: arXiv:2603.20639v1 (5-page essay, March 21, 2026)

  • Key Ideas

  • Reject the monolithic singularity. The standard "AI singularity" picture --- one titanic mind bootstrapping itself to godlike intelligence at a single point --- is "almost certainly wrong in its most fundamental assumption." If AI development follows past major evolutionary transitions, it will be plural, social, and deeply entangled with us.
  • Intelligence is high-dimensional and relational, not a single quantity that must be unambiguously less or greater than human scale. Asking "is the AI smarter than humans?" misframes the question because human intelligence is already collective, not individual.
  • Inside reasoning models there is already a society of thought. Frontier reasoning models (DeepSeek-R1, QwQ-32B) don't improve by "thinking longer"; they spontaneously simulate multi-agent dialogue --- argue, question, verify, reconcile --- within their own chain of thought. Reinforcement learning on reasoning accuracy spontaneously increases conversational, multi-perspective behaviors. RL is rediscovering, by optimization pressure alone, that robust reasoning is a social process.
  • The "cultural ratchet" thesis. Each prior intelligence explosion (primate → human, oral → literate, manuscript → printing) was not an upgrade to individual cognitive hardware; it was the emergence of a new socially-aggregated unit of cognition. LLMs trained on the accumulated output of human social cognition make the cultural ratchet computationally active --- every parameter a compressed residue of communicative exchange.
  • Centaurs, not just agents. "We have entered the era of human-AI centaurs: composite actors that are neither purely human nor purely machine." Each person may move in and out of diverse ensembles many times a day --- one human directing many AI agents; one AI serving many humans; many humans and many AIs collaborating in shifting configurations.
  • Institutional alignment beats RLHF. RLHF is a "parent-child model of correction, fundamentally dyadic and unable to scale to billions of agents." The alternative is borrowed from human political theory (Ostrom, North): persistent institutional templates --- courtrooms, markets, bureaucracies --- defined by roles and norms, with digital equivalents. The identity of any agent matters less than its ability to fulfill a role protocol; "judge," "attorney," and "jury" are well-defined slots independent of who occupies them.
  • AI auditing AI as a constitutional design problem. The authors argue for explicit separation-of-powers architecture: a labor-department AI auditing a corporation's hiring algorithm; a judicial-branch AI evaluating whether an executive-branch AI's risk assessments meet constitutional standards. Compares unfavorably to the scenario "the SEC ineffectively hire[s] business school graduates armed with Excel spreadsheets to combat high-dimensional collusion of AI-augmented trading platforms."

  • No mind is an island (closing line). The intelligence explosion is already here --- in the society of thought inside every reasoning model, in centaur workflows reshaping every knowledge profession, in agent ecologies forking and collaborating at scale. The question is whether we will build social infrastructure worthy of it.

  • Summary

This is a five-page essay, formatted in Google PoI Team / arXiv style, that argues against the dominant "AI singularity" narrative in favor of a plural intelligence explosion. The core empirical move is to point at the recent generation of reasoning models (DeepSeek-R1, QwQ-32B) and observe that what happens inside them is not "one model thinking harder" but "many cognitive perspectives simulating an internal town hall." This emergent multi-agent structure was not designed in --- it appeared spontaneously when RL was applied to reasoning accuracy. The authors argue this is the model rediscovering, under optimization pressure, what social epistemology has long argued: robust reasoning is intrinsically dialogical.

The essay then extrapolates outward. If intelligence is fundamentally social, the path to more powerful AI does not run through a single colossal oracle but through composing richer social systems --- centaurs of humans and agents, agents of agents, recursive societies that fork, collaborate, and reconcile. The dominant alignment paradigm (RLHF) is dyadic and parent-child-shaped, and won't scale. The authors call for institutional alignment --- digital equivalents of courtrooms, markets, bureaucracies, with role protocols and explicit conflict structures. Conflict is a feature, not a bug.

The closing section is the most concrete policy proposal: when AI systems are deployed in high-stakes decisions (hiring, sentencing, benefits allocation, regulatory enforcement), the question of who audits the auditors becomes unavoidable, and the answer must be constitutional in structure. Specific examples: a labor-department AI auditing corporate hiring algorithms; a judicial-branch AI evaluating executive-branch AIs' risk assessments. The cited counter-example --- SEC staff with Excel against high-dimensional AI collusion --- lands.

The references draw on Hannah Arendt-adjacent political theory (Ostrom on commons, North on institutions, the Federalist Papers), team-science (Wuchty/Jones/Uzzi, Mercier/Sperber, Tomasello), and a constellation of arXiv-era AI work, including the authors' own recent papers on "Reasoning Models Generate Societies of Thought" (2026) and "What Do Agents Believe?" (2026). The piece is short but heavy with citations.

  • Relevance to Economics Research

Indirect but conceptually load-bearing. Several reasons it matters for the wiki audience:

  1. Reframes the AGI debate in terms compatible with how economists think. Institutions, role protocols, separation of powers, governance of commons --- this is North/Ostrom/Acemoglu vocabulary, not Bostrom-singularity vocabulary. For economists who have struggled to engage with AI-risk discourse, this paper's framing is the door.

  2. Explicit critique of the SEC's regulatory capacity in the AI era. The "Excel spreadsheets vs. AI-augmented trading platforms" line is a direct prod at financial-regulatory practice. For asset-pricing and microstructure researchers, this is a real problem statement: market-monitoring infrastructure built for human traders and Excel-driven analysts may not survive contact with multi-agent trading ecologies.

  3. Validates the centaur frame that runs through this wiki (Imas's Licklider thread, Hall's "10x research" piece, Cunningham's pipeline, Kustov's posture). The Evans/Bratton/Agüera y Arcas piece is the conceptual high-altitude version of what the practitioner posts have been showing in the trenches.

  4. The "institutional alignment" thesis is testable. If the authors are right that role protocols matter more than agent identity, then the empirical question for finance/governance is: which institutional templates port to digital agents and which don't? That's a research agenda, not just a manifesto.

  5. Related Concepts

  6. concepts/agentic-ai
  7. concepts/ai-agents
  8. concepts/ai-as-normal-technology

  9. concepts/agentic-workflows

  10. Related Summaries

  11. summaries/thread-alexolegimas
  12. summaries/ai-normal-technology
  13. summaries/agentic-everything
  14. summaries/what-will-be-scarce
  15. summaries/ai-people-right-a-lot
  16. summaries/bitter-lesson