What Matters
A crypto model can be mathematically correct and historically useless because the research pipeline leaked the future. The most dangerous leaks come from timestamp mistakes, revised labels, survivor-only universes, random splits, feature normalization and execution assumptions—not from the model architecture itself.
DN Evidence Block
Reviewed: 20 November 2026. Primary evidence: scikit-learn time-series cross-validation, Freqtrade look-ahead analysis, pandas time-series documentation and official blockchain-data interfaces.
Boundary: this workflow reduces common research errors. It cannot prove future profitability or eliminate market-regime change.
Twelve Steps From Raw History to a Defensible Model
| Step | Decision | Leakage risk | Evidence to retain |
|---|---|---|---|
| 1. Question | Define target, horizon and action | Changing target after seeing results | Pre-registered research brief |
| 2. Point-in-time universe | Which assets existed then? | Survivorship and listing bias | Historical listings and delistings |
| 3. Raw data | Collect trades, books, candles and chain data | Provider revisions and missing periods | Immutable snapshots and source |
| 4. Time model | Event time, receipt time and decision time | Using data before it arrived | Availability timestamp |
| 5. Normalize | Units, symbols, contracts and decimals | Future mapping or token collisions | Versioned entity map |
| 6. Labels | Define future outcome without overlap | Label horizon entering features | Label code and cutoff |
| 7. Features | Lag, rolling and cross-sectional transforms | Centered windows and full-sample scaling | Feature availability rule |
| 8. Temporal split | Train, validation and test order | Random mixing of regimes | Cutoffs and gap |
| 9. Costs | Fees, spread, slippage and funding | Mid-price and infinite liquidity | Venue and size assumptions |
| 10. Walk-forward | Retrain using only prior data | One optimized historical window | Each fold and model version |
| 11. Robustness | Stress parameters and data errors | Choosing only successful tests | Full experiment registry |
| 12. Shadow live | Compare current signals with executable outcomes | Assuming backtest equals production | Orders, quotes, latency and rejects |
1. Define the Prediction and the Trade Separately
“Predict Bitcoin” is not a research question. Define the target, horizon, observation time, action and loss function. A model predicting next-hour return is different from one deciding whether to place a limit order that may not fill.
Write the brief before feature exploration. Otherwise researchers can select the target and horizon that make the data look strongest.
2. Reconstruct a Point-in-Time Asset Universe
A current list of tradable tokens excludes projects that failed, delisted or lost liquidity. Training only on survivors makes historical selection easier than reality. Preserve the assets, venues and contracts available at each timestamp.
Use chain ID and contract address rather than symbol alone. Symbols are reused, contracts migrate and wrapped assets create separate instruments.
3. Preserve Raw Data and Provider Versions
Archive the source response, retrieval time, schema and checksum before cleaning. Exchanges correct candles, indexers reprocess events and entity-label providers update classifications. The model must be evaluated on the information available then, not a later corrected dataset unless correction latency is modeled.
4. Separate Event Time From Availability Time
A block event can occur at one timestamp but become available to the strategy later because of confirmations, indexing or provider latency. A news article has a publication time, update time and ingestion time. The feature should enter the model only after it could have reached the system.
For cross-venue data, normalize clocks and retain exchange timestamps and local receipt timestamps. Clock drift can create false lead-lag relationships.
5. Normalize Without Using Future Knowledge
Fit scalers, encoders and imputers on the training window only. Full-sample normalization leaks the distribution of the future. Cross-sectional rankings should use only assets present and observable at that time.
Do not backfill missing values from a later observation. Missingness can itself contain information about outages or illiquidity.
6. Build Labels With a Gap
If the target uses the next 24 hours, examples near the split can overlap the test period. Add a gap or embargo between training and test. Scikit-learn’s TimeSeriesSplit supports a gap parameter, but researchers must choose it based on label and feature horizons.
Label execution outcomes rather than mid-price changes when the intended strategy uses real orders. Include whether a limit order would have filled and at what size.
7. Detect Look-Ahead in Feature Code
Centered rolling windows, negative shifts, dataframe-wide extrema and final-period normalization can quietly access future values. Freqtrade’s lookahead-analysis runs comparison backtests to detect changed indicators and signals, though no automated test can prove the absence of every leak.
Write unit tests that compute each feature using a truncated dataset and compare it with the same timestamp in the full dataset.
8. Use Temporal, Regime-Aware Evaluation
Random cross-validation assumes observations are independent and identically distributed. Markets are autocorrelated and non-stationary. Use expanding or rolling windows where every test period occurs after its training period.
Report performance by bull, bear, high-volatility, low-liquidity and venue-stress regimes. A model that works only in one period is a regime bet, not a universal predictor.
9. Model Execution at the Intended Size
Commission is only one cost. Use bid/ask spread, price impact, order type, partial fills, maker/taker fees, funding, borrow, gas, failed transactions and minimum sizes. Capacity should be estimated from executable depth, not daily reported volume.
10. Keep Every Experiment, Not Only the Winner
Record features, parameters, code commit, data version, random seed, folds and results for all material experiments. The number of trials affects the probability of finding a false winner. A leaderboard without failed experiments hides selection bias.
11. Stress the Model
Delay data, remove a provider, widen spreads, alter fees, perturb labels, shorten history and test adjacent parameters. If performance disappears after a small realistic change, the model is fragile.
12. Shadow Before Funding
Run the production pipeline without executing or with minimal capital. Compare signal timestamp, available quote, intended order, actual fill, latency and resulting position. This is the first point where infrastructure and exchange semantics enter the evidence.
DN Rule
Every feature needs an “available at” timestamp, not merely an “occurred at” timestamp.
DN Crypto Model Leakage Risk Test
Score one research pipeline. Higher scores indicate greater risk that future information entered the result.
Worked Example
A model ranks current top-100 tokens using normalized social sentiment and next-day return. The dataset excludes delisted assets, scales sentiment over the full history and uses random cross-validation. The result shows a high Sharpe ratio.
After rebuilding the point-in-time universe, fitting the scaler per training fold, adding a one-day gap and applying spreads, the edge disappears. The model did not fail; the original research question was never honestly tested.
Methodology and Limitations
DN mapped leakage across universe, timestamps, transforms, labels, splits, costs, trials and deployment. The checklist does not detect every coding error or prove live generalization.
Frequently Asked Questions
What is look-ahead bias in crypto models?
It occurs when training or features use information that would not have been available at the decision timestamp, making historical performance unrealistically strong.
Why is random cross-validation dangerous for market data?
Random splits mix past and future observations and can leak time-dependent regimes, labels and neighboring information into the training set.
How should delisted tokens be handled?
Use a point-in-time universe that includes assets that later failed or delisted. Removing them creates survivorship bias.
Can onchain data leak future information?
Yes. Labels, decoded events, finality, revised indexes and prices may be available later than their blockchain timestamp. Record availability time separately from event time.
When is a model ready for live capital?
After temporal holdouts, walk-forward tests, realistic costs, robustness checks and a shadow or small-capital phase with independent risk limits.
Primary Sources
- scikit-learn TimeSeriesSplit
- Freqtrade Lookahead Analysis
- Pandas Time-Series Documentation
- Dune Documentation
Editorial disclosure: no data provider or framework paid for inclusion.




