A new real-world test puts AI coding agents in a simulated developer job with colleagues, project boards, and incomplete handoffs, where they have to keep developing software on their own and reliably close open tickets in a messy daily workflow. Even Opus 5.5 and Astra fall far short, solving fewer than one in ten tasks completely. The models finish clear, standalone assignments quickly, but they lose track as autonomous developers. The article AI coding agents ace isolated tests but complete fewer than one in ten tasks in a simulated workplace appeared first on The Decoder .
Original text(via The Decoder)
AI coding agents ace isolated tests but complete fewer than one in ten tasks in a simulated workplace
Image description
AI coding agents are nearing top scores on standard benchmarks. A new experiment puts them to work for the first time as digital employees inside a simulated software company with colleagues, project boards, and chat histories. Even Claude Opus 5.5 and GPT-6 Astra fully complete fewer than one in ten tasks.
Success stories about AI coding agents keep piling up. Anthropic says Claude writes more than 80 percent of the company's production code, and standard benchmarks like SWE-bench are nearly saturated.
THE DECODER+
Continue reading with THE DECODER+
Every Plus article and the full archive
No ad banners
Frontier Radar: 6 deep dives a year
Already a subscriber? Sign in Back to the homepage
The text above was extracted automatically from the source page and may miss images, tables or code blocks. The original is authoritative.
Published by The Decoder
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.