What Matters
AI agents do not improve evenly across work. The strongest gains tend to appear where a system can tell whether the task succeeded, identify where it failed and repeat the loop cheaply. That means the most important question for businesses is often not “Is this task complex?” but “Can we verify the result quickly enough to train, evaluate and govern the agent?”
DN Evidence Block
Last reviewed: 15 November 2026.
Directional sources: the supplied transcripts repeatedly emphasize well-scoped tasks, self-checking loops and the difference between work that has objective feedback and work that depends on diffuse judgment.
Primary verification: METR’s November 2026 assessment of AI-driven R&D acceleration at Anthropic; OpenAI’s current agent-evaluation and trace-grading guidance; Anthropic’s evaluator-optimizer workflow guidance.
Boundary: this article does not claim that low-verifiability work cannot be automated. It ranks how easy it is to create a reliable improvement loop around a task.
The Hidden Variable Is Feedback
Two tasks can look equally difficult to a human and behave very differently for an AI system. A coding agent can run tests. A forecasting agent can compare a prediction with the eventual outcome. A reconciliation agent can check whether balances match. A strategy agent deciding whether a brand should enter a new market may have to wait months before anyone knows whether the decision was good.
That gap matters because agent systems improve through repeated evidence. OpenAI’s trace-grading guidance treats an agent run as an inspectable sequence of model calls, tools, guardrails and handoffs. Once those traces can be scored, teams can find regressions and improve the workflow systematically. Anthropic similarly recommends evaluator-optimizer loops when success criteria are clear enough for critique and measurable refinement.
Why This Matters More Than Public Benchmarks
METR’s November 2026 assessment of Anthropic found that the company’s capabilities on well-specified tasks were likely improving at about twice the historical rate, with substantial uncertainty, and judged that a majority of the current acceleration was likely attributable to AI-assisted R&D. METR also explicitly warned that its evidence is strongest for verifiable, well-specified work and that broader tasks are harder to extrapolate.
That qualification is the key. A business should not read “AI progress accelerated” as “every knowledge-work process will improve at the same rate.” The most automation-friendly workflows are the ones where feedback is dense.
The Five Components of Verifiability
| Dimension | High-verifiability example | Low-verifiability example |
|---|---|---|
| Outcome clarity | Invoice reconciles to the ledger | Campaign “feels on brand” |
| Feedback speed | Unit tests return in minutes | New-market strategy plays out over quarters |
| Attribution | One tool call caused one state change | Dozens of decisions interact over time |
| Repeatability | Same input can be replayed in a sandbox | One-off negotiation with unique social context |
| Failure observability | Error code, trace and expected value are visible | Failure appears later as weak growth or reputation loss |
1. Outcome Clarity
A task is easier to improve when the target state can be stated precisely. “Classify this support ticket into the correct queue” is easier to evaluate than “make this customer feel valued.” The latter may still be automatable, but the feedback loop is noisier.
2. Feedback Speed
Fast feedback lets teams run more experiments. A coding loop can write, execute, inspect and repair within minutes. A trading rule can be backtested quickly but still requires caution because historical simulation is not live performance. A sales strategy might require weeks before downstream quality is visible.
3. Attribution
When one action maps cleanly to one effect, an evaluator can tell what worked. When twenty agent decisions interact with human decisions and external conditions, attribution becomes much harder. This is one reason trace-level evaluation is useful: it can isolate whether the wrong tool, handoff or policy choice caused the failure.
4. Repeatability
Replayable tasks are easier to benchmark safely. Historical datasets, sandboxes, test accounts and fixed scenarios let teams compare changes without risking production systems. One-off social or political decisions offer far less repeatability.
5. Failure Observability
An agent can only learn from a failure that the system can recognize. Missing fields, invalid API responses and broken tests are obvious. A subtle decline in brand trust is not.
DN Verifiability Gradient
Score one candidate workflow. This does not predict whether the task should be automated; it estimates how easy the workflow is to evaluate and improve.
12–15: high-verifiability; 8–11: workable with strong eval design; 4–7: weak feedback loop; 0–3: automation claims should be treated cautiously.
What High-Verifiability Does Not Mean
It does not mean “safe,” “important,” or “easy.” High-frequency trading is highly measurable and still extremely risky. Cybersecurity tasks can have objective signals and still face adversarial behavior. Verifiability simply means teams have a stronger basis for knowing whether a system got better.
Where Businesses Should Start
Start with workflows that combine measurable outputs, fast feedback, bounded permissions and replayable test cases. That creates a development flywheel: traces become datasets, datasets become evals, evals reveal regressions, and the agent improves without relying on anecdotes.
Frequently Asked Questions
Does a task need a perfect objective function?
No. It needs enough observable evidence to tell better from worse consistently.
Why are coding agents improving so quickly?
Software provides unusually dense feedback: compilers, tests, static analysis, benchmarks and production telemetry.
Can subjective work still use agents?
Yes, but subjective work needs more human review and more careful evidence about what “good” means.
Is this the same as an AI benchmark?
No. The gradient scores a business process’s feedback structure, not a model’s general intelligence.
Final Verdict
The biggest divide in agentic automation may not be easy work versus hard work. It may be verifiable work versus weakly verifiable work. When the system can check itself, teams can iterate. When it cannot, confidence grows much faster than evidence.
Primary Sources
- METR — Independent Assessment of AI R&D Acceleration at Anthropic, November 2026
- OpenAI — Evaluate agent workflows
- OpenAI — Trace grading
- Anthropic — Building effective agents




