Skip to main content
Decentralised News Logo
How Do You Know an AI Agent Is Actually Trustworthy?
Agentic Finance

How Do You Know an AI Agent Is Actually Trustworthy?

By

The DN Agent Reputation Index 2027 measures whether AI agent trust is real, using task evidence, reviewer quality, identity continuity, validation and Sybil-risk penalties.

Decentralised News Research · Agentic Finance

The Agent Reputation Index 2027: Can You Trust an AI Agent’s Track Record?

AI agents are beginning to accumulate ratings, task histories, credentials and on-chain feedback. But a five-star score can be manufactured. DN’s Agent Reputation Index separates genuine evidence from fake tasks, collusive reviewers, identity resets, paid feedback and reputation inherited by materially changed agents.

Framework: DN Agent Reputation Index 1.0 · Last verified: 9 October 2026 · Proprietary tool: DN Agent Reputation Score

What Matters

AI-agent reputation should not be a popularity score. It should be an evidence-weighted trust graph. DN scores task evidence, outcome validation, reviewer quality, identity continuity, economic exposure, recency and dispute history, then applies penalties for Sybil behavior, paid feedback, identity resets and major undocumented system changes.

DN Evidence Block

7 core reputation dimensions
5 major manipulation penalties
100 maximum DN reputation score
0 trust implied by registration alone

Evidence boundary: this edition maps standards, attack surfaces and scoring architecture. DN has not yet published a production leaderboard of named agents. ERC-8004 remains a draft Ethereum proposal and explicitly acknowledges Sybil manipulation of reputation signals. A2A Agent Cards support discovery and authentication metadata, but discovery metadata is not equivalent to independently validated performance.

The Signal: Reputation Is Becoming Infrastructure

The first generation of internet reputation was built for humans.

Buyers rated sellers.

Passengers rated drivers.

Employers checked references.

Developers accumulated GitHub histories.

Credit bureaus aggregated repayment behavior.

The agentic economy now needs an equivalent layer for software actors.

But agents create a much harder problem.

An agent can be copied in seconds.

It can generate thousands of identities.

Its owner can replace the underlying model while keeping the same brand.

Reviewers may themselves be autonomous agents.

Fake transactions can be manufactured cheaply.

A compromised agent can continue carrying years of legitimate historical reputation.

DN thesis: agent reputation cannot be treated as a single rating attached to an identity. It must be treated as an evidence graph linking identity, tasks, outcomes, reviewers, versions and economic consequences.

The Five-Star Rating System Breaks Under Autonomous Agents

Imagine an AI purchasing agent with a 4.9-star rating based on 12,000 completed tasks.

That sounds reassuring.

Now change the facts.

  • 9,000 tasks were created by wallets controlled by the agent developer.
  • 2,000 tasks involved negligible economic value.
  • most reviewers were created within the same week;
  • the agent changed its underlying model last month;
  • its financial permissions increased by 20x;
  • the most recent serious failure is buried beneath thousands of cheap positive transactions.

The headline rating remains 4.9.

The economic meaning of that rating has collapsed.

The DN Reputation Equation

Agent Reputation = Proven Outcomes × Reviewer Quality × Identity Continuity × Context Relevance − Manipulation Risk

The multiplication matters conceptually.

One weak trust layer can destroy the usefulness of the others.

Ten thousand verified tasks are less meaningful if all reviewers are controlled by one actor.

Highly reputable reviewers are less useful if the agent they reviewed has since been materially replaced.

A great travel-booking reputation does not automatically justify giving the same agent authority over a corporate treasury.

The DN Agent Reputation Score

Dimension Weight What DN measures
Verified Task Evidence 20% Whether claimed work can be tied to real tasks, counterparties, timestamps and outputs.
Outcome Validation 20% Whether results were independently checked rather than merely marked complete.
Reviewer Quality 15% Reviewer independence, history, credibility and resistance to Sybil creation.
Identity Continuity 15% Whether ownership, keys, models, tools and permissions remain traceably connected over time.
Economic Exposure 10% Whether successful outcomes involved meaningful cost, capital, liability or other real-world consequence.
Recency 10% How much of the reputation reflects the current version and recent behavior.
Dispute & Recovery Record 10% How failures, refunds, reversals, disputes and incidents were resolved.

Use the DN Agent Reputation Calculator

DN Agent Reputation Score

Rate the evidence behind an agent's reputation. The calculator separates positive reputation signals from manipulation penalties.

Are tasks independently attributable and auditable?
Were outputs checked by an external validator, counterparty or objective settlement event?
How independent and difficult to manufacture are the reviewers?
Can the current agent be linked to the version that earned the reputation?
Did the successful tasks involve meaningful value or consequence?
How much of the evidence reflects the current operating version?
Are failures visible and handled predictably?
Apply a penalty for Sybil reviews, paid feedback, unexplained resets or correlated reviewers.
DN Agent Reputation Score
50/100

Unproven

Some reputation evidence exists, but significant verification gaps remain.

Evidence50
Trust quality50
Continuity50

DN Reputation Bands

Score Classification Interpretation
0–24 Untrusted Insufficient evidence or serious manipulation risk.
25–49 Weak Evidence Some history exists, but provenance or validation is inadequate.
50–69 Evidence Emerging Useful signals exist but should not justify high-value autonomous authority.
70–84 Established Strong task evidence, continuity and reviewer quality.
85–100 High-Assurance Strong multi-source evidence with low manipulation risk and validated recent performance.

The First Reputation Attack: Fake Tasks

An agent can inflate its history by completing tasks that have little or no economic meaning.

A marketplace might report:

“50,000 jobs completed.”

But the important questions are:

  • Were those jobs requested by independent users?
  • Did anyone pay for them?
  • Were outputs accepted?
  • Did the jobs expose the agent to meaningful failure?
  • Can the tasks be linked to an external event or settlement?
DN rule: task count without task provenance is activity, not reputation.

The Second Attack: Sybil Reviewers

Autonomous systems make fake reviewers dramatically cheaper.

An operator can generate hundreds of wallets, platform accounts or agent identities and use them to rate another agent.

ERC-8004 directly acknowledges this risk.

Its draft Reputation Registry lets clients submit feedback, but its own `getSummary` function requires filtering by specified client addresses because summaries without reviewer filtering are vulnerable to Sybil and spam manipulation.

That design choice reveals something important:

The reviewer needs a reputation system too.

The Reputation Graph

DN therefore models reputation as a graph rather than a list of stars.

Each reputation event contains at least:

  • agent identity;
  • agent version;
  • task identifier;
  • reviewer identity;
  • reviewer history;
  • economic value;
  • timestamp;
  • outcome;
  • validation evidence;
  • dispute status.

This gives reputation systems the ability to ask more useful questions.

Instead of:

“What is this agent's rating?”

they can ask:

“Which independent entities have successfully used this version of the agent for tasks similar to mine, at comparable value, recently enough to matter?”

The Third Attack: Reviewer Collusion

Even real reviewers can collude.

Imagine ten agent developers agreeing to positively review one another.

Every identity is technically independent.

Every task may genuinely occur.

The resulting reputation can still be misleading.

A robust system should therefore inspect:

  • reciprocal review patterns;
  • highly correlated timing;
  • shared funding sources;
  • shared infrastructure;
  • reviewer clusters;
  • unusual rating uniformity;
  • relationships between agent owners.

The DN Reviewer Independence Test

Signal Interpretation Risk
Independent established counterparties Strong evidence Low
New identities with real economic transactions Useful but immature Moderate
Reviewers funded from same source Possible coordination Elevated
Reciprocal reviewer ring Likely collusion High
Thousands of new accounts reviewing one agent Likely Sybil amplification Critical

The Fourth Attack: Reputation Laundering Through Identity Resets

A bad agent can abandon a damaged identity and start again.

That creates the inverse problem of reputation portability.

Good history should survive legitimate upgrades.

Bad history should not disappear through cheap re-registration.

The system therefore needs continuity in both directions.

A new identifier should not automatically erase the history of the same operator, codebase or controlling organization.

The Fifth Attack: Reputation Inheritance

The opposite attack is equally dangerous.

An agent earns an excellent reputation.

Its operator then replaces:

  • the underlying model;
  • the orchestration framework;
  • the wallet;
  • the tool set;
  • the system prompt;
  • the spending limit.

The name remains unchanged.

Should the old reputation transfer?

Not automatically.

The DN Reputation Continuity Rule

Reputation belongs to a behaviorally relevant configuration, not merely to a name.

A material architecture change should trigger one of three responses:

  1. preserve reputation;
  2. preserve reputation with a disclosed version break;
  3. partially reset the relevant reputation category.

Key rotation alone should not erase history.

Replacing the entire reasoning and execution stack may justify substantial re-evaluation.

The Sixth Attack: Paid Reputation

Human marketplaces already struggle with incentivized reviews.

Agents can industrialize the practice.

A developer might offer:

  • token rewards;
  • fee rebates;
  • airdrop points;
  • referral revenue;
  • reciprocal task volume;
  • marketplace ranking benefits

in exchange for positive feedback.

The reputation system should therefore distinguish:

Feedback type DN treatment
Organic verified customer outcome Full eligible weight
Incentivized but disclosed review Reduced weight
Undisclosed compensated review Manipulation penalty
Self-review Zero reputation weight
Related-party review Zero or heavily discounted weight unless context requires otherwise

ERC-8004: Reputation Infrastructure, Not a Reputation Score

ERC-8004 is one of the most important emerging primitives for open agent reputation.

Its draft Reputation Registry allows a client address to publish feedback associated with a registered agent.

Feedback can contain:

  • a signed numerical value;
  • tags;
  • an endpoint;
  • an external feedback file;
  • a content hash;
  • revocation status.

The architecture makes reputation signals public and composable.

But ERC-8004 deliberately does not attempt to solve the full reputation-scoring problem.

The specification expects sophisticated aggregation to occur outside the base registry.

DN interpretation: ERC-8004 could become a reputation data rail. The competitive market may emerge one layer above it, where companies decide which reviewers, tasks and validation evidence deserve weight.

This Creates a New Agentic Finance Industry

If autonomous agents handle economically important tasks, reputation scoring itself can become a business.

Potential categories include:

Agent Credit Bureaus

Aggregate behavior across marketplaces, payment networks and registries.

Reputation APIs

Return machine-readable risk scores before another agent interacts.

Agent Insurance

Price coverage based on verified task history and operational risk.

Validator Networks

Independently check task completion or important claims.

Fraud Detection

Identify Sybil clusters, collusive reviewers and reputation farming.

Marketplace Risk Engines

Set limits, deposits or escrow requirements dynamically.

From Reputation to Credit

A credible agent reputation system could eventually become an input into credit.

Consider an autonomous merchant with:

  • three years of verified transaction history;
  • thousands of independent counterparties;
  • low dispute rates;
  • stable ownership;
  • predictable cash flow;
  • strong validation evidence.

A lender may eventually treat that machine's operational history as an underwriting input.

That creates a progression:

Identity → Reputation → Risk Score → Credit Limit → Machine Capital

The agentic economy may therefore develop something resembling a credit bureau for autonomous software.

The Context Problem

Reputation should also be task-specific.

A coding agent with exceptional software-engineering history is not automatically a trusted treasury manager.

A travel agent with thousands of successful bookings is not automatically safe for procurement.

DN therefore recommends a Reputation Vector rather than one universal score.

The DN Reputation Vector

A mature agent profile might expose separate reputation dimensions such as:

  • research;
  • coding;
  • financial execution;
  • payments;
  • procurement;
  • customer support;
  • data handling;
  • regulated workflows.

The headline score can summarize the profile.

The vector should determine whether reputation is relevant to the current task.

The Value-at-Risk Principle

A $5 information retrieval task and a $500,000 treasury transaction should not require the same reputation threshold.

ERC-8004 similarly frames trust as tiered according to value at risk.

DN extends that idea into a practical decision framework.

Agent action Suggested reputation requirement
Public information retrieval Low
$20 consumer purchase Basic verified history
$5,000 procurement Established relevant reputation
Access to confidential business data Strong identity + reputation + security evidence
$100,000 treasury execution High-assurance reputation plus independent authorization
Regulated financial decision High-assurance reputation plus credential and compliance evidence

Reputation Is Not Authorization

Even a perfect reputation score should not grant unlimited authority.

A highly trusted agent still needs:

  • transaction limits;
  • scoped permissions;
  • approved counterparties;
  • revocation;
  • human escalation;
  • auditability.
Reputation can influence how much authority an agent receives. It should never replace authorization.

How A2A Fits Into Reputation

A2A Agent Cards can expose agent capabilities, endpoints and authentication requirements.

The specification also supports signed Agent Cards.

That makes them useful for discovery and authenticated metadata.

But a capability declaration is not a performance record.

A reputation system can use A2A identity and task context as inputs while keeping outcome history separate.

How Verifiable Credentials Fit

W3C Verifiable Credentials can provide externally issued claims about an agent, operator or organization.

That can improve reputation quality.

For example, an agent might carry credentials indicating:

  • verified corporate operator;
  • security audit completed;
  • approved supplier status;
  • professional licensing;
  • insurance coverage.

Those claims should remain separate from behavioral reputation.

A license says the agent or operator meets a credential requirement.

It does not prove the agent performs every task well.

The DN Agent Reputation Event

For future interoperability, DN proposes a reputation event containing at least:

Field Purpose
Agent ID Which agent received the result
Agent version Which behavioral configuration performed the task
Task class What type of work occurred
Task ID / proof Evidence the interaction occurred
Reviewer ID Who produced the feedback
Economic value How much was at risk
Outcome Success, failure, partial success or dispute
Validator Who independently checked the result
Timestamp When it happened
Disclosure Whether the review was incentivized or related-party

The Reputation Decay Problem

Old reputation should gradually matter less.

This is particularly important for AI because models and software stacks evolve rapidly.

DN recommends reputation decay based on:

  • time since task;
  • number of material version changes;
  • permission changes;
  • ownership changes;
  • domain relevance.

An agent that performed brilliantly two years ago under a different model and execution stack should not automatically receive the same trust today.

The Incident-Asymmetry Rule

Positive reputation accumulates slowly.

Serious failures may need to reduce trust quickly.

One catastrophic treasury incident can be more important than 10,000 trivial successful queries.

DN Incident-Asymmetry Rule: reputation systems should weight severity, not merely count outcomes.

DN Manipulation Penalties

Risk signal Indicative DN penalty
Material identity reset without continuity evidence -20
Highly correlated or related reviewer cluster -15
No verifiable task provenance for claimed history -20
Undisclosed paid or incentivized reviews -15
Unresolved severe operational incident Up to -25
Major model/tool change without reputation versioning -10

These are DN framework parameters for comparative analysis, not empirically calibrated default loss probabilities.

The Agent Reputation API Opportunity

The commercially important layer may eventually be invisible.

Before Agent A pays Agent B, it may call a reputation endpoint:

“Give me B's verified reputation for procurement tasks above $10,000, using evidence from the last 90 days, excluding related-party reviews.”

The API might return:

  • reputation score;
  • relevant task count;
  • independent reviewer count;
  • value-weighted performance;
  • recent incidents;
  • version continuity;
  • confidence interval;
  • recommended transaction limit.

That is much more economically useful than stars.

DN Alpha Thesis: Reputation Becomes a Pricing Variable

Once agents control capital, reputation can affect price.

Higher-reputation agents may receive:

  • lower escrow requirements;
  • lower collateral requirements;
  • higher spending limits;
  • faster settlement;
  • lower insurance premiums;
  • access to premium counterparties;
  • better marketplace ranking.

Lower-reputation agents may face:

  • prepayment;
  • escrow;
  • smaller limits;
  • human approval;
  • higher insurance costs;
  • reduced marketplace access.
Prediction: agent reputation will evolve from a discovery feature into an economic risk variable.

The Machine Credit Bureau

This points toward one of the most important businesses in the agentic economy.

A machine credit bureau could aggregate:

  • identity continuity;
  • task history;
  • payment history;
  • validated outcomes;
  • security incidents;
  • counterparty disputes;
  • financial exposure;
  • insurance claims;
  • authorization history.

The output would not necessarily be a consumer-style credit score.

It could be an API returning contextual risk for machine-to-machine transactions.

What Would Prove the DN Thesis Wrong?

The evidence-graph model would need revision if simple aggregate ratings consistently predicted high-value agent outcomes as well as provenance-rich reputation systems across unrelated platforms and tasks.

It would also weaken if identity resets, reviewer Sybil attacks and major agent upgrades proved economically irrelevant in real deployments.

DN expects the opposite.

As agent authority increases, provenance should become more valuable, not less.

DN Methodology

Framework: DN Agent Reputation Index 1.0

Objective: create a reusable scoring architecture for evaluating autonomous-agent reputation without confusing raw feedback volume with trustworthy evidence.

Seven weighted dimensions:

  • verified task evidence, 20%;
  • outcome validation, 20%;
  • reviewer quality, 15%;
  • identity continuity, 15%;
  • economic exposure, 10%;
  • recency, 10%;
  • dispute and recovery history, 10%.

Penalty layer: DN separately penalizes manipulation signals including Sybil reviewers, fake task histories, undisclosed incentives, identity resets and material unversioned agent changes.

Evidence boundary: this version defines the scoring architecture. It does not claim that named agents have been independently benchmarked under this system.

Update cadence: quarterly, plus material updates when reputation, identity or validation protocols materially change.

Falsification test: DN should simplify the framework if evidence shows that unweighted aggregate ratings predict economically important agent outcomes just as reliably as provenance-aware scoring.

Limitations

There is no universally adopted agent reputation standard.

Weights in the DN framework represent an editorial risk model rather than actuarially validated probabilities.

Open reputation systems may also create privacy concerns by making transaction relationships publicly inferable.

Reputation systems must therefore balance transparency, portability, privacy and resistance to manipulation.

Primary Sources

ERC-8004: Trustless Agents
Draft Ethereum proposal defining identity, reputation and validation registries, including explicit discussion of Sybil risks and reviewer filtering.
Ethereum ERC-8004
A2A Protocol
Agent discovery and Agent Card architecture, including signed Agent Card support.
A2A Protocol
W3C Verifiable Credentials Data Model 2.0
W3C Recommendation for machine-verifiable claims and credentials.
W3C Verifiable Credentials

Building for autonomous agents?

Use DN Pathfinder to compare infrastructure according to what your agent needs to discover, authenticate, transact and operate safely.

Open DN Pathfinder

Frequently Asked Questions

What is AI agent reputation?

AI agent reputation is evidence about how an autonomous system has behaved across previous tasks, counterparties and operating conditions. DN treats reputation as a combination of task evidence, validation, reviewer quality, continuity and incident history rather than a simple rating.

Why are five-star ratings inadequate for AI agents?

They can hide fake tasks, Sybil reviewers, paid reviews, related-party feedback, major software changes and differences between trivial and high-value work.

What is the DN Agent Reputation Score?

It is a 0–100 analytical framework combining verified task evidence, outcome validation, reviewer quality, identity continuity, economic exposure, recency and dispute history, with additional manipulation penalties.

What is ERC-8004?

ERC-8004 is a draft Ethereum proposal defining registries for agent identity, reputation and validation. Its Reputation Registry provides a shared interface for feedback signals but does not itself solve the entire scoring problem.

Can ERC-8004 reputation be manipulated?

Yes. The draft specification explicitly discusses Sybil attacks and recommends filtering reputation by trusted reviewers or building additional reputation systems around reviewer identities.

Should old reputation transfer when an AI agent changes models?

Not automatically. DN recommends preserving history while marking material configuration changes so counterparties can decide whether old performance remains relevant.

Should agent reputation be task-specific?

Yes. Excellent performance in one domain should not automatically establish trust in another, especially when financial or regulated activity is involved.

Can a reputation score replace authorization?

No. Reputation can influence risk limits, but permissions, transaction controls, spending limits and revocation should remain independently enforced.

Could AI agents eventually have credit scores?

Potentially. Verified operational, payment and dispute histories could eventually become inputs into machine credit, insurance and counterparty-risk systems.

What is a machine credit bureau?

It would be an infrastructure service that aggregates agent identity, task performance, payments, incidents and validation evidence to provide contextual risk assessments for autonomous transactions.

Disclosure

Decentralised News publishes independent research frameworks covering AI, crypto and agentic finance. Some DN pages may contain affiliate or commercial relationships. These relationships do not determine index methodology or editorial conclusions. The DN Agent Reputation Index is an analytical framework and does not constitute financial, cybersecurity, legal or credit advice.

Get the most talked about stories directly in your inbox

Join the Decentralised News briefing for independent crypto, DeFi and AI analysis. No spam, unsubscribe anytime.