A task horizon is not a stopwatch
METR’s task-completion time horizon estimates the duration a human expert would need for tasks an AI agent can complete at a stated reliability level. It measures task difficulty using human-expert completion time. It does not measure how long an agent process remains open. Keep these measures separate:
A twelve-hour run may contain a long-horizon task, but duration does not prove it.
Compare two jobs:
- Long batch: summarize one thousand unrelated documents.
- Long-horizon task: investigate, design, build, test, and explain one product change where every stage depends on the work before it.
Task structure
Two kinds of long-running work
Long-running batch
Many separate outputs
- Independent units
- No shared state required
- Failures stay local
Long-horizon task
One definition of done
- Dependent stages
- Decisions carry forward
- Recovery uses saved state
The horizon is the chain of dependency, not the stopwatch.
What makes an AI agent task long-horizon?
Long-horizon work usually has:- one coherent outcome;
- dependent stages;
- accumulated state and decisions;
- feedback that changes the next action;
- several checkpoints or context windows;
- delayed but observable verification;
- one shared definition of done.
Examples of long-horizon agent tasks
Suitable examples include:- migrating a large system while preserving behavior;
- debugging a coupled numerical or production failure;
- building and validating a complete product slice;
- conducting research where each experiment changes the next experiment;
- producing coordinated code, design, documentation, screenshots, demo material, and launch assets for one product goal;
- investigating and implementing a customer-specific workflow in a forward-deployed engineering engagement.
What does not qualify?
These may run for a long time without becoming long-horizon tasks:- a very long prompt;
- one slow build, query, training job, or command;
- a large batch of independent classifications or summaries;
- a recurring report that starts fresh each time;
- a chat that remains open overnight;
- unrelated agents producing disconnected outputs.
Why long-running AI agents are becoming practical
Current agent products combine stronger models with execution systems that preserve an objective, use tools, delegate selected work, inspect results, and return artifacts for review. OpenAI positions GPT-5.6 Sol for complex, open-ended work. Goal mode gives a Codex task a persistent objective, constraints, and completion criteria. Ultra can use subagents for separable work. Anthropic positions Claude Fable 5 for ambitious, long-running asynchronous projects. In an agent harness such as Claude Code, it can plan across stages, delegate selected work, check its results, and continue for extended sessions. Neither system guarantees a production-quality outcome. The result still depends on the task, context, execution environment, feedback, verification, and human review.Why agents lose the plot over long work
Anthropic’s long-running agent harness research documents recurring failures:- attempting the entire project in one pass;
- exhausting the context window in the middle of a change;
- leaving partial work undocumented;
- repeating approaches that already failed;
- reporting completion after partial progress;
- testing implementation details without validating the user flow;
- allowing parallel agents to collide on shared dependencies.
How to prepare a long-horizon task
Give the agent a durable operating record instead of relying on the initial prompt and chat history.
Keep the environment recoverable. Use version control, isolated branches or worktrees, repeatable setup commands, and checkpoints that another session can inspect. Each stage should leave the project clean enough for the next stage to continue without reconstructing the work.
Anthropic’s research on long-running Claude recommends well-scoped work, explicit success criteria, progress memory, a test oracle, Git coordination, and occasional human oversight. Its examples also produced test gaps and mistakes, which is why the returned evidence still needs review.
Should one agent work sequentially or use subagents?
Use subagents for separable work such as independent research, test analysis, asset exploration, or a review of a finished change. Keep tightly coupled work sequential when every decision changes the downstream system. One main agent can preserve the objective and project state while delegating bounded tasks. Several agents can work in parallel when their outputs have clear interfaces and isolated workspaces. Do not treat more agents as proof of a longer or better task. Parallel agents that edit the same dependencies can create collisions and contradictory decisions.What should the agent return?
Completion should mean ready for informed human review. A useful handoff can include:- the authorizing goal and exact version;
- exact code revisions;
- a map of the consequential changes;
- validation results and anything skipped or failing;
- screenshots, recordings, traces, and interactive explainers;
- failed approaches and unresolved questions;
- proposed updates to living documentation.
The Night Shift is one team pattern
The AI agent Night Shift gives long-horizon work uninterrupted time between two periods of human attention. In the first run described there, a/goal task using GPT-5.6 Sol with Ultra continued for more than twelve hours and returned coordinated product, design, development, image, and go-to-market work. Teammates reviewed those outputs the next day and helped set the next goal.
The schedule is not the definition. The durable objective, dependent stages, accumulated state, verification, and review boundary make the work long-horizon.
Where Plannotator fits
Open-source Plannotator does not run or schedule long-horizon agents. It lets you review the plan before execution and review the returned changes afterward. Plan version history and revision diffs help you check what changed between feedback rounds.Frequently asked questions
How long must a task run to count as long-horizon?
There is no universal wall-clock threshold. Long-horizon tasks have sustained dependencies across their stages. Formal evaluations may report a human-expert task-completion horizon at a particular success rate.What is the difference between long-horizon and long-running?
Long-horizon describes the structure and difficulty of the task. Long-running describes the execution workflow that keeps an agent working through it.Does long-horizon mean fully autonomous?
No. An agent can work asynchronously inside strict permissions, stop for important decisions, and return its result for review.Do long-horizon tasks require subagents?
No. Subagents help with separable work. Tightly coupled tasks may be better served by one sequential agent that delegates only selected tasks.Can Codex and Claude perform long-horizon tasks?
OpenAI and Anthropic provide models and agent workflows for long-running, multi-step work. Results still depend on the task, execution system, context, verification, and review boundary.Should a long-running agent merge or deploy its own work?
Not by default. Integration and deployment authority are separate decisions with their own permissions and blast radius.Run a bounded team Night Shift
See how teams approve one goal, let agents work asynchronously, and review the returned code and artifacts together.