The DN Alpha Thesis: context has a carrying cost

An agent may request a small fact from a tool and then repeatedly carry that response through later model calls. Add the original instructions, previous outputs, tool descriptions and a growing conversation history, and the input consumed across the run can substantially exceed the unique information supplied by the user.

DN calls this the context carrying cost: the token exposure created by repeatedly making state available during a workflow. It is not automatically waste. Some repeated state preserves requirements and prevents mistakes. The research question is which information needs to remain active, which can be retrieved on demand and which can be replaced with a verified summary.

Measuring the largest individual prompt misses that question. A series of modest calls can consume more aggregate input than one large call. An apparent compression saving can disappear when the agent performs extra searches, retries or summary repairs.

This index therefore evaluates the entire declared workflow. It complements the Agent Model Efficiency Index, which focuses on successful work per dollar, and the memory benchmark, which tests retained state. Here the denominator is aggregate input-token consumption.

What the primary evidence tells us

Lost in the Middle evaluated multi-document question answering and key-value retrieval. In its studied settings, performance often depended on where relevant information appeared, with weaker use of information in the middle of the context. This historical result motivates position-sensitive tests; it does not prove identical behavior for every 2026 model or agent. [1]

Anthropic’s context-engineering guidance discusses curating instructions, tool information, retrieved data and history. It describes just-in-time retrieval, compaction and structured notes, while warning that aggressive compaction can discard important details. This is provider engineering guidance rather than an independent comparative benchmark. [2]

DN’s inference is practical: evaluate both removal and recovery. A compact representation is useful only if it preserves the information needed to meet the acceptance rule or provides a reliable way to retrieve it.

The DN Context Efficiency metric

Sum model input usage across every call inside the system boundary. Include input occurrences served from cache, counting each occurrence once; do not add a cached subset twice if already included in the total. Include compression, routing and summary calls when they are part of the evaluated system. Do not count an output token again as output here; count its later occurrence if it becomes input to another call.

Internal processing that is not exposed in usage cannot be reconstructed from this metric. Record the provider’s usage semantics and the included components. Monetary cost should come from actual billing records or clearly labelled estimates, rather than assuming all input tokens share one price.

A fictional improvement that still needs investigation

Suppose the baseline attempts 100 tasks, accepts 90 and consumes one million input tokens at a total cost of 20 currency units. A candidate attempts the same tasks, accepts 88 and consumes 500,000 input tokens at a cost of 12.

Baseline context efficiency is 9 accepted tasks per 100,000 input tokens. Candidate efficiency is 17.6. Token consumption falls 50%, and cost per accepted task falls from about 0.2222 to 0.1364. But acceptance also falls from 90% to 88%.

The candidate is more efficient by the proposed token metric and less successful by the task acceptance measure. Investigate the lost outcomes before deployment. An omitted contractual requirement or permission boundary can be more consequential than the saving. None of these numbers is an observed model result.

Compare context strategies on the same contract

No provider is ranked, promoted or assigned an operational status in this edition. Costs depend on the actual implementation and service terms. For agentic finance, preserve authorization and transaction state outside an unverified narrative summary; the context strategy itself does not establish custody or execution authority.

A test pack that exposes information loss

Choose a fixed set of tasks and define success before running them. Use an independent evaluator and preserve the original evidence. Keep model, tokenizer, tools, decoding settings and task order controlled where possible. Change one context strategy at a time.

Count all input consumed until final evaluation, not just the first request. Keep failed tasks in the denominator. Record median and tail latency separately; token reduction alone cannot establish faster delivery. Repeat enough runs to understand variability and publish sample counts rather than inventing confidence from a single clean demonstration.

What belongs in the evidence register

Each comparison needs task IDs, model and tokenizer versions, strategy configuration, source snapshots, per-call usage, cached-input treatment, total cost categories, acceptance outcomes and critical failures. Preserve the evaluator version and the exact system boundary.

Record how much context is submitted and which components are reintroduced: instructions, tool schemas, source evidence, tool results, history and structured state. That component ledger helps identify where to investigate. It does not prove that every repeated token is unnecessary.

Keep measurement and interpretation separate. “Input fell by 50%” can be a measured statement. “Half the context was waste” requires an additional causal argument showing that the removed information was unnecessary under the tested conditions.

Methodology, limitations and falsification

Method: review primary long-context research and provider engineering guidance; define accepted outcomes per aggregate input consumption; propose paired fixtures and quality gates; calculate diagnostics from reader-entered totals. No live model benchmark or vendor leaderboard has been performed.

Dataset status: the published fixture register is a proposed testing dataset, not observed performance data. The worked example is fictional. Future reports should publish sanitized task fixtures, per-call usage summaries, denominators and results on a permanent versioned URL.

Limits: tokenizers differ, task difficulty differs and stochastic outputs vary. Caching changes billing without necessarily changing logical input exposure. The metric does not measure semantic relevance, inaccessible internal compute or total infrastructure efficiency. Cross-model rankings need explicit caveats and additional outcome/cost measures.

Falsification test: reject a claim of useful context optimisation if matched evaluation finds lost necessary evidence, unacceptable quality decline, critical control failures or hidden auxiliary costs that reverse the claimed economic benefit.

Maintenance: rerun after model, tokenizer, tool, retrieval or compaction changes. Review sources quarterly. Change log: 2 October 2026, initial DN-CEI methodology and calculator. Corrections: send the claim, fixture and evidence through DN’s contact page.

Sources

1. Liu et al.: Lost in the Middle: How Language Models Use Long Contexts, Transactions of the Association for Computational Linguistics, 2024.
2. Anthropic: Effective context engineering for AI agents, 29 September 2025.

Frequently asked questions