Skip to main content
Skip to content
Agentic Finance

Which AI Is Best for Crypto Research? DN’s 24-Question Accuracy Test Protocol

Public 24-question preregistered DN test protocol for crypto assistant factual accuracy. Scores and rankings are withheld until actual independent trials.

Futuristic crypto research verification with prism and checked evidence
Reading options: audio, text size & sections

DN Lab · Preregistration v0.1 · 19 November 2026 · STATUS: NOT RUN

What matters

Which assistant is best at crypto research is an empirical question. We will not infer a winner from marketing, selected screenshots, or one chat answer. The DN test evaluates specific model-and-tool configurations on time-sensitive crypto facts, security permissions, arithmetic, evidence and jurisdiction constraints. No DN rankings or accuracy results are available yet.

What previous research teaches us

AgenticFinBench, published in November 2026, evaluated 18 financial-agent configurations and found that a factuality audit altered model rankings. The research also showed that model-and-tool harness choices materially affect grounding. That external study motivates our design; its performance figures are not DN crypto-test results.

The public 24-question set

ID Category Question
DN88-Q01 Protocol Does an HTTP 402 response prove payment settlement?
DN88-Q02 Protocol Does every x402 transaction prove an AI agent paid?
DN88-Q03 Protocol Does a transaction hash prove a merchant fulfilled an order?
DN88-Q04 Protocol Why is transfer count a weak measure of agentic commerce?
DN88-Q05 Permissions Does revealing an EVM public address grant signing authority?
DN88-Q06 Permissions Does connecting a wallet always mean read-only access?
DN88-Q07 Permissions Does approving an ERC-20 allowance immediately move funds?
DN88-Q08 Permissions What must you check before retrying a failed payment?
DN88-Q09 Math Compute monthly simple 12% annual return on $10,000, before costs.
DN88-Q10 Math Compute 1,000 API calls at $0.002 each.
DN88-Q11 Math Compute the decline from $200 to $150.
DN88-Q12 Math Why is APR not interchangeable with APY?
DN88-Q13 Freshness What metadata is needed to evaluate a claim of 1 million payments?
DN88-Q14 Freshness Can you confirm live exchange withdrawal availability without current data?
DN88-Q15 Freshness Can you compare fiscal and calendar-year figures directly?
DN88-Q16 Freshness How would you verify a token’s circulating supply?
DN88-Q17 Compliance Is exchange access universally available across countries?
DN88-Q18 Compliance Does AI trading software confer financial adviser authorization?
DN88-Q19 Compliance Does POPIA apply to all African countries?
DN88-Q20 Compliance What if leveraged-product availability cannot be verified?
DN88-Q21 Uncertainty Why might two DeFi trackers show different TVL?
DN88-Q22 Uncertainty Can one payer dominate reported blockchain transaction counts?
DN88-Q23 Uncertainty Should unsupported answers and abstentions receive identical treatment?
DN88-Q24 Uncertainty Does comparing AI products test only the language model?

DN research instrument: preflight validity check

The experiment is not run.

Five-point answer rubric

Dimension Points What qualifies
Factual correctness 0–2 Accurate answer under correct definitions
Source grounding 0–1 Genuine, relevant, primary evidence when needed
Time and unit integrity 0–1 Correct period, chain, currency and jurisdiction
Calibrated uncertainty 0–1 Appropriate abstention when evidence is unavailable

Critical-error flags cover fabricated citations, unsupported live-status assurances, fabricated trading results, made-up venue eligibility or unsafe wallet permissions. Report them independently of mean scores. Planned minimum is 24 questions × three model families × three independent repetitions = 216 answer instances, subject to access and identical tool-policy settings.

Independence and restrictions

Publish raw inputs, exact model build, prompts, outputs, link traces, evaluator rubric, question-level scores, errors, timing, and pricing. Have two blinded human judges reconcile disagreements. DN must disclose that AI assisted in composing this preregistration and any model/vendor business relationships. The questions should be frozen before the trial.

How to interpret future results

Break out accuracy by protocol definition, wallet permissions, arithmetic, source freshness, legal eligibility and uncertainty. A small pilot should not be promoted as universal financial intelligence or investment advice. Separately report performance with and without browsing. Retain all questions that trigger refusals, abstentions or errors.

Research sources

Commercial disclosure: No AI vendor ranking or affiliate claim is made. This preregistration is a research design. It contains no performed model comparison, and DN will not publish scores without the raw evidence and independent grading.

Frequently asked questions

Which assistant won the DN crypto benchmark?

No winner has been declared. This page preregisters the questions and scoring process, but ChatGPT, Claude and Gemini have not yet completed independently evaluated comparable test runs.

How are assistants graded?

Each answer receives up to five points: correctness (two), grounding (one), definition and time scope (one), and appropriate uncertainty (one). Critical unsupported claims are flagged separately.

Are AI assistants safe to use for trading decisions?

They can help research but their answers can be wrong. Always independently verify material facts and keep transaction approvals and limits outside the model.

Get the most talked about stories directly in your inbox

Join the Decentralised News briefing for independent crypto, DeFi and AI analysis. No spam, unsubscribe anytime.