Definition
The effective number of trials is a modeling summary of how many independent or weakly dependent opportunities a selection process resembles. It is used when the literal count of tried variants overstates, under the displayed bounded-count convention, the independent-search burden. There is no universal formula: the dependence structure and selection protocol determine the defensible estimate.
How it is calculated
Effective trial count m-eff is a specified dependence-adjusted count that is no greater than the literal trial count m in this convention.
| Symbol | Meaning | Unit |
|---|---|---|
| m | literal number of trials | count |
Interpretation
Trying twenty nearly identical variants is not the same search as trying twenty unrelated ideas. Effective trials compress that difference into a quantity useful for calibration. The compression is an assumption to defend, not a number that can be read from a final Sharpe ratio.
Assumptions and conventions
-
The trial population includes the variants that influenced selection.
-
The dependence or correlation structure is modeled or justified.
-
The effective count is used only within the adjustment that defined it.
Count of effective trials.
Variants
- The trial population includes the variants that influenced selection.
Limitations and common misreadings
-
Effective trials are not a universal correction factor.
-
Ignoring unreported trials biases the adjustment toward optimism.
References
- David H. Bailey, Marcos López de Prado. (2014). The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting, and Non-Normality. The Journal of Portfolio Management. Discussion of estimating independent trials from correlated trials for the deflated Sharpe ratio. doi:10.3905/jpm.2014.40.5.094. Source

