- METR's 80% time horizon lags the 50% horizon by about eight months. I had been erroneously reading that lag as a task's transition time, the time a task takes to go from a 50% to an 80% success rate.
- However, in METR's data the median task's transition time is about four months, half the lag, because tasks of the same duration get solved at different times.
- That spread among tasks of the same duration also matters for other readings of the horizons, for example:For hazard rates, the pooled data can show a falling hazard rate even if every task has a constant one.For labor displacement, if automating a role needs every task done reliably, the timing depends on the hardest tasks of each duration, which are solved later than the horizon suggests.
- For hazard rates, the pooled data can show a falling hazard rate even if every task has a constant one.
- For labor displacement, if automating a role needs every task done reliably, the timing depends on the hardest tasks of each duration, which are solved later than the horizon suggests.
This may be obvious to others, and in hindsight it is to me too, but I needed a closer look at the data to see it.
The 80% horizon trails the 50% horizon by about eight months
In February 2025, the frontier model Claude 3.7 Sonnet had a 50% time horizon of one hour. The 80% horizon reached one hour about nine months later. Across the models since GPT-4 that moved the 50% horizon forward, the median gap is about eight months (90% interval 6.4 to 10.8).
| Model | 50% horizon | First model with an 80% horizon past it | Estimated months later |
|---|---|---|---|
| GPT-4 | 4 min | o1-preview | 17.7 |
| GPT-4o | 7 min | o1 | 6.7 |
| Claude 3.5 Sonnet (old) | 11 min | Claude 3.7 Sonnet | 7.9 |
| o1-preview | 20 min | o3 | 6.4 |
| o1 | 39 min | Gemini 3 Pro | 8.2 |
| Claude 3.7 Sonnet | 60 min | GPT-5.2 | 9.3 |
| o3 | 2.0 h | Claude Mythos Preview | 10.8 |
First model with an 80% horizon past it
Estimated months later
Claude 3.5 Sonnet (old)
Claude 3.7 Sonnet
Gemini 3 Pro
Claude 3.7 Sonnet
Claude Mythos Preview
The frontier 80% horizon is interpolated between model releases.
The horizon is an average over tasks
For each AI model, METR fits one success curve across all tasks, with human completion time as the difficulty. A 50% horizon of two hours says that a randomly chosen two-hour task is completed half the time.[1] In practice, tasks of the same duration differ a lot in how hard they are for models.
A familiar example is children learning to walk. One child in ten walks alone at 10 months and nine in ten at about 14 and a half, a span of four to five months. Yet a single child's transition from crawling to walking takes only a number of weeks (Adolph and Tamis-LeMonda, 2014).
Figure 1. Share of children walking alone, by age
The blue curve is from the WHO Motor Development Study. The gray lines are illustrative children.
METR's time horizon is an average over tasks in the same way. Tasks of the same duration are solved at different times. Among two-hour tasks, a quarter are as easy for models as a typical 40-minute task and a quarter as hard as a typical six-hour task. Each task goes from unsolved to solved quickly, but the group's success rate rises slowly.
The five example tasks in Figure 2 all take a human five to eight hours but behave like typical tasks of 27 minutes to 35 hours. The hardest is still failed on every attempt, and the other four each took about four months to go from a 50% to an 80% success rate.
Figure 2. Five tasks that take a human five to eight hours
Each column is one frontier model at its release date, and each block is one attempt.
Across the whole suite, a frontier model gets mixed results on a task, passing some attempts and failing others, in 16% of cases. If every task followed METR's fitted curve for its duration, mixed results would be expected in about 40% of cases.[2]
Figure 3. Results are mixed in 16% of cases, against 40% predicted by METR's curves
Each case is one frontier model's attempts at one task, typically four.
The transition time of a single task
To estimate how fast a single task is solved, I fitted a model with a difficulty and a steepness for each task. The median transition time is about four months (90% interval 3.7 to 4.3). It is under three months for a quarter of tasks and over five for another quarter.
That is about half the eight-month gap between the horizons, and only about 6% of tasks take as long as the gap. The gap is longer because two random tasks of similar duration are typically solved by the frontier about seven months apart, and a quarter of such pairs more than a year apart.
Figure 4. Single tasks flip quickly. Groups of tasks decline slowly because they flip at different times.
Chance of failing against calendar time, using the ability of the best model released so far. Dots are raw run outcomes for the whole band.
As an illustration, going from a 10% to a 90% success rate (roughly, from the first occasional successes to nearly always solved) takes the median task about a year. On METR's curves, the 90% horizon reaches a given task length a bit over two years after the 10% horizon.
These single-task estimates need a different model from METR's, which takes a task's difficulty to be its human duration. I fitted a hierarchical Bayesian item response model, close to the one in Jonas Moss's post.[3] It differs from METR's in two ways.
- Each task gets its own difficulty and steepness. The difficulty starts at the task's human duration and moves as far as the results support. Steepness is how sharply the task goes from unsolved to solved as models get more capable.
- Each AI model gets an adjustment for each task family, because its results on tasks of the same family are correlated beyond what its ability and the tasks' difficulties explain.
The model gives a doubling time of 4.4 months for the frontier, against 4.1 months from METR's horizons.
What an 80% horizon means for automation
Forecasts such as the AI Futures Project's treat a long enough 80% time horizon as the point where AI can replace software engineers. But an 80% success rate on two-hour tasks could come from every task failing one time in five, in which case you just retry. Or it could come from one task in five failing every time, however often you retry.
METR's data are somewhere in between. For each frontier model I counted the tasks between half and twice its 80% horizon, and how many of them it failed in every run. Until Mythos that was 0 to 14% of them. With Mythos it jumped to 26% (8 of 31 tasks).
| Frontier model | Released | 80% horizon | Tasks near it | Failed in every run |
|---|---|---|---|---|
| Claude 3.7 Sonnet | Feb 2025 | 12 min | 37 | 5 (14%) |
| o3 | Apr 2025 | 30 min | 36 | 0 |
| GPT-5 | Aug 2025 | 38 min | 32 | 0 |
| Gemini 3 Pro | Nov 2025 | 54 min | 30 | 0 |
| Claude Opus 4.5 | Nov 2025 | 49 min | 34 | 3 (9%) |
| GPT-5.2 | Dec 2025 | 66 min | 28 | 2 (7%) |
| Claude Opus 4.6 | Feb 2026 | 70 min | 29 | 1 (3%) |
| Claude Mythos Preview | Apr 2026 | about 3 h | 31 | 8 (26%) |
Frontier model
Tasks near it
Failed in every run
Claude 3.7 Sonnet
Gemini 3 Pro
Claude Opus 4.5
Claude Opus 4.6
Claude Mythos Preview
Hazard rates
The hazard rate is the chance, per minute of a task, that an agent fails it. Ord proposed that it is constant for a given agent, and Hamilton fits METR's results and finds that it falls as tasks get longer (a Weibull shape of about 0.6, where 1 means constant). With a constant rate, success falls off exponentially with task length. With a falling rate, failure gets less likely the further an agent has got into a task, and long tasks are easier than a constant rate predicts.
Both readings take a task's human duration as its difficulty. But tasks of the same duration differ in difficulty. When you pool tasks that each have their own constant hazard rate, the pooled rate looks like it is falling. In a simulation where every task has a constant hazard rate, the pooled fit still gives Hamilton's shape.
| Results | Weibull shape of the pooled fit |
|---|---|
| METR's results | 0.59 |
| Simulated: constant hazard rate for every task, with the difficulty spread estimated in this post | 0.57 |
| Simulated: constant hazard rate, with duration as the only difficulty | 1.01 |
Weibull shape of the pooled fit
METR's results
Simulated: constant hazard rate for every task, with the difficulty spread estimated in this post
Simulated: constant hazard rate, with duration as the only difficulty
Median across frontier models.
Appendix
Recap of METR's time-horizon methodology
For each AI model, METR fits a logistic regression of success on the log of the task's human time. In item response theory (IRT) terms, this is a model in which task difficulty is kept fixed and each AI model gets an ability and a slope of its own. The most widely cited time horizon figure reports the ability estimates for frontier models in the original unit, human time.
Why METR fixes task difficulty
METR sets each task's difficulty to its human time. I think that is a sensible choice, for two reasons.
- Fixed difficulties stay the same whichever models are evaluated. Fitted difficulties, like the ones in this post, can shift when a new model is added, and the reported horizons would shift with them.
- Fitting a difficulty for every task takes much more data. Each model attempts a task only a handful of times, and for a task that every model solves, or that none does, the data can only bound the difficulty.
By letting task difficulties vary, I can see in return how each task behaves on its own, including how quickly it goes from unsolved to solved.
The model
Each AI model m has an ability , and each task i has a difficulty and a steepness . Abilities and difficulties are on the same scale, the log of human time. A model solves a task half the time when its ability equals the task's difficulty, and the steepness sets how quickly the success probability rises as ability passes that point.
The ability of each AI model is estimated from its own results, with a weak prior. The task parameters are partially pooled. A task's difficulty has a prior centered on its human time, and its estimate is shrunk toward that value when the data say little. A task's steepness is drawn toward the typical steepness when its own attempts carry little information. Each AI model also has an adjustment for each task family. Without it, the fitted model's errors on two tasks of the same family have a small but measurable correlation (0.17, against zero across families).
All three models fit , where is the human time of task i and is its family.
| METR | Moss | This post | |
|---|---|---|---|
| Task difficulty | , fixed | , with , , estimated | |
| Steepness | one per AI model, | one per task, | one per task, |
| AI model ability | one per AI model, from that model's own regression | for release date , so models are pulled toward a time trend | independently, a weak prior with no time trend |
| Task families | each task weighted by | none | a model-by-family term added to the ability, , with |
| Fitting | each AI model separately | all models and tasks together | all models and tasks together |
| Uncertainty | bootstrap over families, tasks and runs | posterior intervals | posterior intervals |
Task difficulty
, with , , estimated
one per AI model,
one per task,
AI model ability
one per AI model, from that model's own regression
for release date , so models are pulled toward a time trend
independently, a weak prior with no time trend
Task families
each task weighted by
a model-by-family term added to the ability, , with
each AI model separately
all models and tasks together
bootstrap over families, tasks and runs
posterior intervals
Model comparison
I fitted the model with its parts added one at a time and scored each version by five-fold cross-validation over (AI model, task) pairs. About 17% of the variance in single attempts is run-to-run variance, the differences between repeated attempts by the same AI model on the same task. The remaining 83% is the most any of these models can explain.
| Model | Held-out variance explained | Gain in held-out log score over the previous row | Median transition time |
|---|---|---|---|
| METR's model, with human duration as the difficulty | 53% | ||
| + a difficulty for each task | 72% | +3,228 (se 132) | 6.0 months |
| + a steepness for each task (the simplest model that gives task-specific transition times) | 73% | +349 (se 48) | 4.6 months |
| + an AI model by family term (the model in this post) | 74% | +365 (se 51) | 4.0 months |
Held-out variance explained
Gain in held-out log score over the previous row
Median transition time
METR's model, with human duration as the difficulty
- a difficulty for each task
+3,228 (se 132)
- a steepness for each task (the simplest model that gives task-specific transition times)
+349 (se 48)
- an AI model by family term (the model in this post)
+365 (se 51)
A difficulty for each task accounts for most of the gain. Steepness and the AI model by family term each add about one point of explained variance or less, because most pairs pass or fail on every attempt and these terms only change predictions for the rest. Both improve the held-out log score by about seven standard errors, and the AI model by family term moves the median transition time from 4.6 to 4.0 months.
Transition time by task duration
Figure A1. Transition time of each task, grouped by task duration
Each dot is one task's estimated time to go from a 50% to an 80% success rate. The line is the group's median and the band its middle half. Hover a dot for the task name.
Transition times are similar across task durations, with medians between about three and four and a half months, and slightly shorter for tasks over an hour. Within each group most tasks fall between two and six months. Single-task estimates are uncertain and are pulled toward the typical value, so the spread in the figure is narrower than the true spread.
- ^METR's FAQ says the answer is somewhere in between a model doing half of all two-hour tasks and doing each of them half the time. On tasks of 90 minutes to three hours, GPT-5 always succeeds on about a third, always fails on about a third and is mixed on the rest.
- ^Nguyen and Fithian report the same pattern. They find a model's attempts at a task are mixed in 17% of cases, against about 40% predicted by METR's curves. Their model also estimates a difficulty for each task and each family. It assumes every task goes from unsolved to solved at the same pace, and the model here estimates that pace for each task.
- ^Moss shows that METR's 80% horizons shift a lot once difficulty varies among tasks of the same duration. He ties each AI model's ability to a trend over time and studies where the horizons are heading. This post asks how long a single task takes to be solved.