Skip to content

Can AI Replace Social Science Researchers?

Key Ideas

  • The peer-reviewed journal article as a primary unit of academic production is likely dead, and that may be a good thing -- it was already a flawed metric system.
  • The publish-or-perish system incentivized quantity over quality, and AI makes it trivial to produce "barely publishable" articles at scale, accelerating an already-unstable equilibrium.
  • If you think Claude Code is a better social scientist than you, the problem is that you stopped trying to answer interesting questions and started optimizing for publication metrics.
  • AI can be genuinely useful as a research tool (e.g., automating data collection, building and maintaining datasets), but the core of social science is asking and answering interesting questions.
  • The bigger crisis facing academia is not AI but the defunding and political assault on higher education and academic freedom.
  • Researchers should focus on "What questions do I actually want to answer?" rather than chasing journal metrics that are collapsing anyway.

Summary

David Karpf responds to viral claims about AI replacing academic researchers by arguing that the real issue is not AI capability but the broken incentive structure of academic publishing. He agrees that the journal article as the primary unit of academic production is probably dead, but frames this as the collapse of an already-dysfunctional system that prioritized counting publications over producing genuine knowledge. The publish-or-perish treadmill created incentives for "barely publishable" work, and AI simply accelerates that system's breakdown.

Karpf draws on C. Thi Nguyen's work on metrics and games to argue that journal articles were always just a scorecard, not the actual purpose of social science. He acknowledges productive uses of AI in research -- he describes his own early career experience building a political blog tracking system and notes that AI would have made the automated portions much easier -- but insists that staying close to the data and understanding what it tells you remains essential. He emphasizes that AI is a useful tool for specific tasks like dataset construction and maintenance, not a replacement for the intellectual core of research.

The article concludes by arguing that the AI story is secondary to the larger crisis: decades of defunding higher education, increasingly precarious academic labor, and political attacks on universities and researchers. These structural problems interact with AI in destructive ways, making it harder for the academic community to thoughtfully integrate new tools.

Relevance to Economics Research

Karpf's argument challenges economists to distinguish between the game of publishing (where AI excels) and the substance of knowledge production (where human judgment remains essential). His point about broken metrics is especially relevant for economics, where publication counts and journal rankings heavily determine careers. The article also raises important questions about how tenure and promotion committees should evaluate research output in an era of AI-assisted production.