Evaluation Gate · Bailey et al. (2016)

Does This Strategy Have
a Real Edge, or Just Luck?

Before any signal reaches deployment, it must pass four independent statistical gates. Each test addresses a different type of false discovery risk: from data leakage and overfitting to pure randomness. All four must pass simultaneously.

Gate FAILED4 / 4 criteria3 Monte Carlo tests
Overall verdict
FAIL
Deflated SR✗ not available
PBO Score✗ not available
Monte Carlo✗ 0 / 3
What each test is actually checking
🔍
Leakage Audit
Risk: Data contamination

Checks that no signal uses future information; 45-day 13F lag + T+1 fill delay enforced throughout

🧬
Provenance
Risk: Missing data bias

Checks CUSIP→ticker resolution rate. Below 90% means too many signals have no real price data

🔁
Reproducibility
Risk: Randomness masquerading as skill

SHA-256 checksums across 3 independent runs confirm the output is deterministic, not stochastic luck

📊
PBO
Risk: Backtest overfitting

CSCV measures what fraction of in-sample winners lose out-of-sample; directly quantifies overfitting risk

Gate Conditions

All four must pass simultaneously. Failure on any single check blocks signal deployment.

🔍
Leakage Audit
No lookahead bias in signal construction
✓ PASS

0 errors, 0 warnings across 86 signals.

🧬
Provenance Threshold
CUSIP→ticker resolution rate must exceed 90%
✓ PASS
ScoreThreshold
82.1%90% limit

23/28 CUSIPs resolved to a real ticker.

🔁
Deterministic Reproducibility
SHA-256 checksums must match across all runs
✓ PASS

Single run; independent re-run cross-check not performed.

📊
Probability of Backtest Overfitting (PBO)
CSCV overfitting score must stay below 40%
✗ FAIL

Not enough trades in this run to compute a PBO score.

Deflated Sharpe RatioBailey & Lopez de Prado (2014)

The observed Sharpe is a biased statistic when you've tested multiple configs. DSR applies four simultaneous penalties (multiple testing, skewness, fat tails, and serial correlation) to produce a conservative, publication-grade estimate. A DSR above 1.0 means the edge survives all adjustments.

Not enough trades in this run to compute a Deflated Sharpe Ratio (minimum 10 required).

n Trials Adj.
not available
Skewness
not available
Excess Kurtosis
not available
Serial Corr.
not available
Benchmark SR
not available
Significant
NO
Probability of Backtest OverfittingBailey et al. (2016)

CSCV splits the backtest into 16 equal partitions and evaluates all 12,870 C(16,8) combinations. For each, it asks: does the in-sample best strategy also win out-of-sample? PBO is the fraction of cases where it does not. Below 40% = acceptable.

Not enough trades in this run to compute a PBO score (minimum 80 required).

CSCV Partitions
16
Combinations
12,870

Monte Carlo Robustness Tests

Three independent null hypothesis tests, each with N=1,000 simulations. Each asks a different question: "Could this result have been generated by chance?" All three must return p < 0.05.

🔄Bootstrap
Is the Sharpe stable across trade samples?

Resamples trades with replacement 1,000 times. If the observed Sharpe is in the right tail of the null distribution, timing is not the explanation.

🎲Random Entry
Does signal timing actually matter?

Randomises entry dates across 1,000 runs. If the real strategy significantly outperforms, signal timing has genuine predictive value.

🌀Regime Permutation
Is the regime multiplier real?

Shuffles regime labels across 1,000 runs. If the real RACS significantly outperforms, the HMM macro signal adds genuine value, not just label noise.

Bootstrap
p = 0.824 · threshold p < 0.05
✗ NOT SIGNIFICANT
Observed SR -0.594
Null 5th %ile
-1.740
Null Median
-0.598
Null 95th %ile
0.530
Observed SR
-0.594
Random Entry
p = 0.995 · threshold p < 0.05
✗ NOT SIGNIFICANT
Observed SR -0.594
Null 5th %ile
-0.594
Null Median
-0.594
Null 95th %ile
-0.594
Observed SR
-0.594
Regime Permutation
p = 0.338 · threshold p < 0.05
✗ NOT SIGNIFICANT
Observed SR 0.020
Null 5th %ile
0.000
Null Median
0.007
Null 95th %ile
0.055
Observed SR
0.020

Walk-Forward Validation

No expanding-window folds recorded for this run.

This run does not span enough calendar years to form a 5-year train / 1-year test fold. More filing history needs to flow through the pipeline before walk-forward validation can run.