The AI People Have Been Right a Lot

  • Author/Source: Dylan Matthews (Vox / Future Perfect), Substack

  • Original: https://dylanmatthews.substack.com/p/the-ai-people-have-been-right-a-lot

  • Key Ideas

  • In 2015 Matthews wrote a Vox article warning that effective altruism was being "nerdsniped" by speculative AI risk, before the transformer existed and before OpenAI was founded. Eleven years later he says, plainly, that he was wrong --- "extremely, extremely obvious that I was wrong."
  • The people who took AI seriously then were not united by a single forecast --- they were united by openness to the possibility that AI would be a huge deal. That openness, not any specific prediction, was what produced career bets that paid off (Chris Olah → mechanistic interpretability + Anthropic; Amanda Askell → Claude's personality; Buck Shlegeris → Redwood Research).
  • Heuristics that misled him, and lessons:
    1. Trusting "mainstream institutions" to detect transformative changes is a mistake --- they aren't as good at prediction as he assumed, and may not even be incentivized to predict rigorously. (COVID is the cited counter-example.)
    2. A bias against "futurism" --- treating any argument that requires multiple imaginative leaps as frivolous --- is widespread (especially in academia) and was wrong in this case.
    3. Specific community track record matters: Ajeya Cotra and Peter Wildeford's end-of-2024 predictions for 2025 were "very, very accurate" (errors mostly on the side of underestimating AI revenue). Aschenbrenner's Situational Awareness (2024) was mostly right; his projection of $520B AI infrastructure spend for 2026 is tracking $650--700B.
  • Updated heuristic: when the AGI-pilled crowd predicts something wild, default to engagement-with-skepticism rather than outright dismissal. He still doesn't expect 30% YoY economic growth, but he won't dismiss the case for it.

  • The Keith Gessen / Edward Keenan anecdote frames the spirit: "You could say to Edward Keenan, and people did say, You're crazy! But no one would be fool enough to say to Edward Keenan, You're a fool." Take seriously, even when you disagree.

  • Summary

Matthews's piece is a public update --- a journalist with no incentive to recant doing it anyway. The structure is honest: in 2015 he attended his first EA Global, was alarmed at how much oxygen AI risk was taking, wrote a Vox article saying so, lost the argument with Scott Alexander, and concluded he was probably still right. He now says: he was wrong. Not wrong in the specific sense of failing to adopt one forecast or another, but wrong in the posture of dismissing a community that turned out to be unusually good at predicting both the trajectory and the magnitude of AI progress.

The most useful part for working researchers is the diagnosis of the heuristics that produced the error. Three are widely shared --- "mainstream institutions would be reacting if it mattered," "futurism is unrigorous," and "I will trust empiricism over speculation" --- and all three failed here. The track record observation is concrete: Cotra, Wildeford, and Aschenbrenner each made specific predictions that were closer to right than the consensus, and the residual gap is on the side of underestimating.

The post ends with the Gessen/Keenan parable: take seriously even what you think is wrong. That posture, applied retroactively, would have changed Matthews's 2015 behavior; applied prospectively, it changes how he reads the 30% YoY growth predictions and the short-timeline forecasts now.

  • Relevance to Economics Research

Indirect but worth filing. For economists thinking about whether to invest substantial career capital in AI-related research, Matthews's diagnosis of the prior he had against AI ("a grumpy empiricist's bias against futurism") is widely shared in the discipline. The Aschenbrenner / Cotra / Wildeford track record is also useful as a calibration anchor when reading current short-timeline forecasts. Kustov in summaries/academics-wake-up-3 cites this exact piece as one of the indicators that the AI conversation has reached "even the most thoughtful people in journalism."

For Mihail's master-class audience, this is a useful counterweight to the discipline's natural skepticism: the people who said this would be a big deal were right, more often than the consensus said they would be. That doesn't validate every prediction they make now, but it raises the prior we should put on ones that sound wild.