Newshot

An LLM workflow that reproduces, improves, extends published economics research

Original text(via HN AI)

An LLM Workflow That Reproduces, Improves, and Extends Published Economics Research

We introduce an open-source workflow that enables an LLM to reproduce, improve, and extend an economics article using the article’s published replication package. First, the workflow attempts to reproduce the original calculations, checks for discrepancies with published findings, and performs automated sensitivity analysis. Across 4,452 published replication packages for five economics journals, the workflow flags discrepancies in 3,460 articles or their appendices. Second, the workflow improves the original calculations by using a different implementation or algorithm. In 496 articles, the workflow is able to reduce a calculation’s computation time, at similar or greater accuracy, by more than a factor of 10. Third, the workflow extends the original analysis. In 923 articles, the workflow develops an extension that does not appear in the original article and that is aligned with the original article’s goals and assumptions.

  • LLMs were used in the analysis and writing of this article. For this project, Schwartz worked as a contractor for Anthropic; the results and views are not endorsed by Anthropic. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research. Jesse M. Shapiro Shapiro has, in the past, been a paid visitor at Microsoft Research New England and a paid consultant for FutureOfCapitalism, LLC. Shapiro has been paid for writing by the New York Times. Shapiro's spouse has a disclosure statement posted at https://vivo.brown.edu/display/eoster1#Affiliations.

LLMs were used in the analysis and writing of this article. For this project, Schwartz worked as a contractor for Anthropic; the results and views are not endorsed by Anthropic. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.

Shapiro has, in the past, been a paid visitor at Microsoft Research New England and a paid consultant for FutureOfCapitalism, LLC. Shapiro has been paid for writing by the New York Times.

Shapiro's spouse has a disclosure statement posted at https://vivo.brown.edu/display/eoster1#Affiliations.

  • Copy Citation Matthew Schwartz, Isaiah Andrews, and Jesse M. Shapiro, "An LLM Workflow That Reproduces, Improves, and Extends Published Economics Research," NBER Working Paper 35782 (2026), https://doi.org/10.3386/w35782. Download Citation
  • Location for public releases of project materials

Related

Topics

Programs

More from the NBER

In addition to working papers, the NBER disseminates affiliates’ latest findings through a range of free periodicals—the NBER Reporter, the NBER Digest, the Bulletin on Health, the Bulletin on the Economics of Alzheimer's, and the Bulletin on Entrepreneurship—as well as research spotlights, lectures, and panel videos.

2026, 18th Annual Feldstein Lecture, Mark Duggan, "Social Security Reform in an Era of Fiscal Deficits" Primary tabs
2026, 18th Annual Feldstein Lecture, Mark Duggan, "Social Security Reform in an Era of Fiscal Deficits" Primary tabs
  • Feldstein Lecture
  • Presenter: Mark Duggan
SI 2026 Methods Lecture
SI 2026 Methods Lecture
  • Methods Lectures
  • Presenters: Melissa Dell & Ashesh Rambachan
Using AI in Household Finance Research: A Practical Guide
Using AI in Household Finance Research: A Practical Guide
  • Presenter: Paul Goldsmith-Pinkham

The text above was extracted automatically from the source page and may miss images, tables or code blocks. The original is authoritative.

Published by HN AI
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗