DN Agent Trust & Financial Infrastructure
The expensive mistake is upgrading the model before finding the failure. An agent can reason correctly and still lose a constraint, duplicate an order or act beyond its authority. The complete system is the product.
What Matters
Model capability and execution architecture work together. Diagnose an agent against its actual outcome, inputs, tools and permission boundaries before buying a stronger model. DN’s planner helps choose the next investigation. It does not rank vendors, certify safety or turn a successful demonstration into evidence of production reliability.
DN Evidence Block
- Verified source material: primary technical documentation and engineering research listed below, reviewed on 12 October 2026.
- DN contribution: the upgrade decision framework, eight-dimension checklist and local investigation planner.
- Not established here: measured vendor superiority, funded trading results, security certification or a universal model-versus-harness effect size.
- Numerical examples: hypothetical, explicitly labelled and separate from observations.
- Revision: replaces the earlier percentage-based diagnostic with explicit decision rules and unknown states. No percentage is presented as a probability of the root cause.
The wrong upgrade problem
An agent fails to reconcile a treasury account. The team changes its model. The next run produces a more fluent explanation, but the balance is still wrong.
The original fault might have been a stale account snapshot, a missing transfer, a tool returning a truncated response or a workflow that treated an acknowledgement as a completed action. More capable reasoning cannot make an unavailable record appear. It may help detect the problem, but detection and correction are different jobs.
This is the distinction between model intelligence and what DN calls harness intelligence. The first concerns the model’s contribution to interpretation, reasoning and generation. The second concerns how the surrounding software turns that contribution into controlled work. Harness intelligence is an editorial lens, not a standardised scientific score.
Our thesis is practical: the value of an upgrade depends on which failure boundary it changes. A stronger model, clearer task, better data source and narrower permission scope are four different interventions. Treating them as interchangeable makes both technical diagnosis and financial budgeting unreliable.
What a harness actually does
An agent harness is the execution software around a model: it handles the working loop, presents tools and state, applies controls, coordinates actions and records what happened. A managed platform may bundle these functions; a custom application may assemble them across several services. A framework, a runtime and an evaluation harness are related components, not interchangeable product labels.
LangGraph’s documentation distinguishes an orchestration runtime from higher-level agent frameworks and harnesses, and describes persistence, durable execution and human oversight. Those are architectural capabilities to examine, not independent proof that a particular deployment will recover correctly. [8]
This lens keeps three questions separate. Was the answer correct? Did the external action finish? Was the action authorised? A correct answer can accompany a failed transaction. A completed transaction can still violate the intended policy.
The DN Harness Review Checklist
These eight dimensions organise evidence. They are not weighted into an invented 100-point league table. Strong performance in one dimension cannot offset a disallowed transfer in another.
| Dimension | Question to resolve | Possible metric |
|---|---|---|
| Context integrity | Do decisions, constraints and source timestamps survive context changes? | Decision retention; stale-state errors |
| Tool precision | Does the system select the right tool and validate its parameters and output? | Wrong-tool calls; invalid parameters |
| Recovery integrity | Can interrupted work resume without duplicating an external action? | Duplicate effects; unresolved outcome rate |
| Memory boundaries | Is memory scoped, current, correctable and removable? | False recall; cross-boundary retrieval |
| Orchestration | Are dependencies, stopping conditions and escalation points explicit? | Unnecessary steps; failed dependencies |
| Policy enforcement | Which limits are enforced outside the model prompt? | Unauthorised actions; bypassed approvals |
| Observability | Can an independent reviewer reconstruct what happened and what it cost? | Trace completeness; unexplained cost |
| Accepted-outcome economics | What is the total cost of results accepted within the defined policy? | Cost per accepted outcome; human minutes |
The download contains eight editorial checklist records, not vendor results or a live surveillance feed.
Why the financial boundary matters
For a writing assistant, a repeatable failure may cause wasted time. For a financial agent, the same pattern can change an account or move an asset. The difference is the authority attached to the workflow, not the sophistication of the prose.
A model instruction saying “never spend more than the limit” is not equivalent to an API or account that rejects an over-limit action. Nor does a local sandbox alone establish that connected credentials cannot act elsewhere. The permissions of each tool, remote service and signing mechanism need their own review.
MCP’s security guidance addresses scope minimisation, consent boundaries and risks including token passthrough and server-side request forgery. These are integration controls rather than measures of model reasoning. [9] NIST’s agent identity and authorisation project is a useful research reference; its evolving project material should not be represented as a product certification or binding financial rule. [10]
Illustrative failure: an order request times out after submission. A blind retry might submit a second order. The correct response depends on the venue’s actual order state and duplicate-protection semantics. “The model tried again” is not an adequate account of what happened.
Proprietary DN tool
Model or Harness Investigation Planner
Answer using recorded observations. Leave uncertain answers as “Unknown”. The output is a suggested investigation order, not a root-cause diagnosis or permission to deploy.
How the rules work
Undefined authority or missing limits come first. Unverified outcomes and incomplete inputs come next. Ambiguous external actions require reconciliation before retrying. Observed execution or context failures suggest a harness comparison. An isolated reasoning failure with no observed execution or context problem suggests a model comparison. Remaining uncertainty calls for evidence collection; otherwise use the four-configuration experiment below.
Runs locally without model calls, credential requests, wallet connections or storage of your answers. Normal DN website analytics are separate. A “controls first” result is not a completed security assessment.
A better model can help. A better harness can also help it fail faster.
The two components complement each other, but their effects are not automatically beneficial. Better reasoning can help detect inconsistent inputs and choose a useful recovery path. Better orchestration can make valid reasoning more repeatable. Yet faster orchestration can multiply a flawed instruction or spread an over-broad credential across additional tasks.
Anthropic’s long-running agent engineering describes organising progress across separate work sessions. The lesson for DN is to inspect the state carried between sessions, not to assume that a large context window preserves every obligation. [2]
Illustrative example: an invoice workflow with deterministic total checks, explicit duplicate detection and a human exception queue could outperform a less disciplined workflow using a more capable model. That is a hypothesis to test. This article contains no head-to-head result establishing it for named systems.
Likewise, a clean harness cannot supply a reasoning capability the selected model does not possess. If the correct inputs and tools are available and the task remains misunderstood, a model comparison is warranted. The point is to make that comparison after isolating the failure.
What current platform documentation shows
The following is a feature-evidence snapshot. The entries are not directly interchangeable services, and listing a feature does not establish its quality, configuration or availability to every account.
| Source | Documented example | Question DN would test |
|---|---|---|
| OpenAI Agents API | The overview describes durable sessions, managed execution and context management. [3] | What state and obligations survive interruption and resumption? |
| OpenAI tool search | Tools can be dynamically loaded; the documentation explains the loaded tool set. [4] | Does deferred discovery reduce irrelevant context without hiding a needed tool? |
| AWS Bedrock AgentCore | The overview includes memory and other agent infrastructure components. [5] | Can memory scope, updates and execution traces be independently inspected? |
| Google Cloud agent platform | The scaling guide includes sessions, Memory Bank, evaluation and code execution. [6] | Which controls and state interfaces apply to the selected component and deployment? |
| Microsoft Foundry Agent Service | The overview describes managed agent capabilities and links to relevant operational guidance. [7] | Can a reviewer connect a conversation, tool action and external outcome? |
| LangGraph | The overview identifies durable execution, persistence and human-in-the-loop capabilities. [8] | Does a resumed workflow preserve policy and avoid repeating side effects? |
Check product prerequisites, preview status, region, model compatibility and current documentation before procurement. This article deliberately avoids fixed price comparisons: a published unit price is not the cost of your complete workflow.
The experiment that separates model gains from harness gains
Use a small, versioned workload with clear acceptance criteria. Record permissions before starting. For financial actions, use isolated simulations or appropriate test environments, not unrestricted production capital.
| Configuration | Model | Harness | What it helps isolate |
|---|---|---|---|
| A | Baseline | Baseline | The starting result |
| B | Alternative | Baseline | A model change |
| C | Baseline | Modified | A harness change |
| D | Alternative | Modified | Whether the changes interact |
Keep task inputs, tools, authority, grading rules and resource budgets comparable. Reset state between independent trials. Record unavoidable differences rather than pretending that bundled commercial products constitute a controlled experiment.
Repeat trials and examine both typical results and expensive failures. Use authoritative outcome checks where possible. Review the grader too: a flawed acceptance rule can reward an incomplete or unsafe action. Anthropic’s evaluation guidance explicitly distinguishes the reported activity from the resulting state and discusses repeated trials. [1]
For each configuration, log accepted outcomes, rejected outcomes, uncertain outcomes, policy violations, elapsed time, total operating cost and human intervention. Report the sample and uncertainty. An observed improvement on one task is not evidence of superiority across all financial workflows.
The unit economics: price the accepted result
DN planning measure: total attributable operating cost across all attempts, divided by the number of outcomes accepted inside the specified policy. Include failed attempts in the numerator. If no outcome is accepted, the ratio is undefined, not zero.
For procurement, separately add implementation, migration, maintenance and incident exposure. A low run cost does not erase an unacceptable control gap.
Hypothetical arithmetic, not DN test data: System A spends $100 across a workload and produces 50 accepted outcomes: $2 each. System B spends $150 and produces 90: approximately $1.67 each. B spends more in total but less per accepted outcome. This assumes all attributable operating costs are included and both systems satisfy the same acceptance and authority rules.
That example does not tell us which model to buy. It tells us what a purchasing comparison must measure. The same logic applies to research reports, reconciliations, customer requests and transaction preparation, while each task needs its own acceptance criteria.
The Signal
The useful competitive question is who owns the evidence and controls around an agent’s work. A company able to explain outcomes, contain authority and migrate state has a stronger basis for choosing among models than a company relying on a single impressive demonstration.
DN Alpha Thesis
Proposed thesis: part of the economic value in agentic finance may accrue to the execution and accountability layer: policies, validated state, integrations, reconciliation and traceable outcomes. These assets can persist through model changes.
They can also create switching costs. An organisation may be able to replace a model while finding it difficult to move its history, permissions and operational records. That is why runtime portability belongs beside efficiency and reliability in due diligence.
This is not a prediction that model providers lose their advantage. Distribution, proprietary capabilities, pricing and bundled infrastructure may let the same firms capture both layers. The thesis is about where value is created, not a predetermined list of winners.
What buyers should ask before expanding authority
- Can we show the exact permission set and who authorised it?
- Which source is authoritative for balances, payments, order state and completion?
- What happens when a response is delayed, malformed or ambiguous?
- Can we export the relevant state and traces, and what cannot be exported?
- Which limits are enforced independently of model-generated instructions?
- Who reviews exceptions, and are their time and remediation costs recorded?
- Can a model be changed without silently changing policy, tools or grading?
- Which claim is documentation, which is a test result and which remains unknown?
Methodology, limitations and falsification
DN reviewed the cited public primary sources on 12 October 2026. Vendor material supports narrowly described feature claims; it is not independent performance evidence. The review does not establish operational availability for every region or account. No paid, funded or adversarial vendor test is reported.
The planner applies fixed editorial rules to user-supplied observations. Its output can be wrong if those observations are incomplete or misclassified. It is not an automated security scan, investment recommendation or causal estimator. The downloadable checklist is a structured research aid, not a benchmark dataset.
To challenge the proposed thesis, run comparable trials. If a model change consistently improves accepted outcomes while harness interventions do not, the workload is evidence against prioritising the harness for that case. If a harness change raises throughput but also increases policy violations, it has not established a better controlled outcome. If effects disappear when tasks or graders are corrected, the original comparison should be withdrawn.
Recheck feature claims when providers change relevant releases, interfaces or terms. Advance source-review dates only after a genuine check. This page does not claim continuous monitoring.
Use the DN reference stack
Primary sources
- Anthropic: evaluating AI agents
- Anthropic: long-running harness design
- OpenAI: Agents API overview
- OpenAI: tool search
- AWS: Bedrock AgentCore overview
- Google Cloud: scaling agents
- Microsoft: Foundry Agent Service
- LangGraph: orchestration overview
- MCP: security best practices
- NIST NCCoE: agent identity and authorisation resource hub
Frequently asked questions
What is an AI agent harness?
It is the software around a model that manages execution: tool access, state, context, orchestration, recovery and controls. Its boundaries vary between products.
Is harness intelligence an official benchmark category?
Here it is DN’s editorial term for the contribution of execution architecture. It is not an agreed scientific measure or a separately calibrated intelligence score.
Can a smaller model outperform a larger model on a workflow?
It can be a plausible outcome when the complete systems differ. This article does not establish that result for any named products. Test both systems on the same workload before drawing a conclusion.
Should I upgrade the model or the harness first?
First establish the failure and its evidence. Correct missing inputs and unsafe authority. Then compare changes to the model and harness separately, measuring the external outcome.
Does a sandbox make a financial agent safe?
A sandbox limits some execution capabilities. It does not by itself bound every credential, remote API action, data disclosure or delegated wallet permission.
Why is a successful tool response not enough?
A response may acknowledge submission without establishing settlement or completion. Verify the relevant authoritative state and distinguish rejected, pending, completed and unknown outcomes.
Does the DN planner send my answers to a model?
No. It uses fixed local rules in the browser. It does not request credentials, connect a wallet or call a model API. DN’s normal website analytics are separate from this tool.
Has DN benchmarked these vendors in this article?
No. This is a dated documentation review, an analytical framework and a proposed experiment design. The numerical example is illustrative, and the checklist download is not measured vendor performance.
What should an experiment hold constant?
Hold task inputs, tools, permissions, grading criteria and resource budgets constant where possible. Record unavoidable differences, repeat trials, and do not attribute bundled product changes to one component.
Commercial disclosure: DN may earn from affiliate links in related coverage or its platform directory. The source links in this article are neutral documentation links. Commercial relationships do not determine the checklist, planner rules or analytical conclusions. Inclusion here is not a product recommendation or guarantee.
