Sign up
ContentsHideShow

Quant glossary · Research & validation

Effective number of trials

The effective number of trials summarizes how many independent or weakly dependent opportunities a selection process resembles.

Definition

The effective number of trials is a modeling summary of how many independent or weakly dependent opportunities a selection process resembles. It is used when the literal count of tried variants overstates, under the displayed bounded-count convention, the independent-search burden. There is no universal formula: the dependence structure and selection protocol determine the defensible estimate.

How it is calculated

meff≤mm_{eff}\le m

Effective trial count m-eff is a specified dependence-adjusted count that is no greater than the literal trial count m in this convention.

Symbol Meaning Unit
m literal number of trials count

Interpretation

Trying twenty nearly identical variants is not the same search as trying twenty unrelated ideas. Effective trials compress that difference into a quantity useful for calibration. The compression is an assumption to defend, not a number that can be read from a final Sharpe ratio.

Assumptions and conventions

  • The trial population includes the variants that influenced selection.

  • The dependence or correlation structure is modeled or justified.

  • The effective count is used only within the adjustment that defined it.

Count of effective trials.

Variants

  • The trial population includes the variants that influenced selection.

Limitations and common misreadings

  • Effective trials are not a universal correction factor.

  • Ignoring unreported trials biases the adjustment toward optimism.

References

  1. David H. Bailey, Marcos López de Prado. (2014). The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting, and Non-Normality. The Journal of Portfolio Management. Discussion of estimating independent trials from correlated trials for the deflated Sharpe ratio. doi:10.3905/jpm.2014.40.5.094. Source

Effective number of trials FAQ

What is Effective number of trials?

The effective number of trials summarizes how many independent or weakly dependent opportunities a selection process resembles.

How is Effective number of trials calculated?

Effective trial count m-eff is a specified dependence-adjusted count that is no greater than the literal trial count m in this convention.

What does Effective number of trials not tell you or assume?

Effective trials are not a universal correction factor.

Browse all glossary termsResearch and validation termsQuant research library

Definitions are educational and descriptive, not investment advice. Worked examples use labelled sample data.