Lab

Lab · backtest overfitting

Deflated Sharpe Ratio calculator

How likely is a backtest's Sharpe ratio to be real, once the configurations tried to find it are counted? A dependency-free Python package and CLI, checked against every example in the source papers.

A backtest arrives as a Sharpe ratio, a sample length and little else. Deflating it asks a different question: how often would a search of this size produce a number this good with no skill behind it. Fill in the backtest below and every quantity updates as you type.

Deflated Sharpe Ratio calculator

examples
Return frequency
1,250 observations

Length bounds

minimum track record to beat zero0.67 y
minimum backtest length for 100 trials1.02 y

The second is the data needed before a search that size stops producing 2.50 by chance.

Bailey and López de Prado (2014): a treasury seasonality backtest, best of 100 configurations. Published answer: SR₀ 0.1132 per day and DSR 0.9004, short of the 95% bar.

Deflated Sharpe Ratio

0.9004

below the 95% bar

Undeflated PSR

1.0000

probability the true Sharpe ratio beats zero, before the search is counted

Selection threshold SR₀

1.79

annualised, against 2.50 observed

Trials this result survives

46

at 95% confidence

Track record needed to pass

8.21 y

holding the trial count at 100

Sharpe ratio needed to pass

2.71

annualised, same sample and trials

0.000.250.500.751.0095% bar1101001,00010,000100,000distinct trials, log scale

Reproduce this result in the package

both snippets track the inputs above

Command line
dsr deflate --sharpe 2.5000 --annual --periods-per-year 250 --n-obs 1250 --trials 100 --trial-variance 0.5000 --skew -3.0000 --kurtosis 10.0000
Python
from deflated_sharpe import deflated_sharpe_ratio

result = deflated_sharpe_ratio(
    sharpe=0.15811388,   # per period
    n_obs=1250,
    n_trials=100,
    var_trials=0.0020000000,
    skew=-3.0000,
    kurtosis=10.0000,
)
result.dsr   # 0.9004

The mathematics

All quantities are per period: a daily Sharpe ratio is the mean daily return over its standard deviation, and T counts daily observations. The Probabilistic Sharpe Ratio (Bailey and López de Prado 2012) is the probability that the true Sharpe ratio exceeds a benchmark SR*, allowing for the skewness γ₃ and raw kurtosis γ₄ of the returns:

If N strategies with no skill are tried and their estimated Sharpe ratios have variance V, the best of them is expected to show

where γ ≈ 0.5772 is the Euler–Mascheroni constant. The Deflated Sharpe Ratio is the Probabilistic Sharpe Ratio with that expected maximum as the benchmark, , so it corrects for the search and for non-normal returns at the same time.

Two length bounds follow from the same algebra. The minimum track record length is the number of observations at which PSR reaches a chosen confidence 1 − α, and the minimum backtest length is the number of years of data before N trials stop producing an annualised Sharpe ratio of E[max] by chance (Bailey, Borwein, López de Prado and Zhu 2014):

Those four expressions produce every number in the calculator above. It opens on the worked example in Bailey and López de Prado (2014), an annualised Sharpe ratio of 2.5 over five years of daily returns, chosen from 100 configurations whose Sharpe ratios have variance 0.5, with skewness −3 and kurtosis 10. That backtest reaches 0.9004 and falls short of the 95% bar, while the same returns would have passed as the best of 46 trials.

Why a Sharpe ratio needs deflating

A researcher who tries a hundred versions of a strategy and reports the best one has run a hundred experiments and published the luckiest. Even when none of the versions has any skill, the best of them shows a respectable Sharpe ratio while a longer search produces an even better-looking winner. The Deflated Sharpe Ratio of Bailey and López de Prado (2014) estimates the probability that the true Sharpe ratio of the selected strategy is above zero once that search is taken into account.

The demo below makes the problem concrete. Every strategy in it has a true Sharpe ratio of exactly zero, yet the best of sixty usually shows an annualised Sharpe ratio above 1.

Demo: the best of N strategies with no skill

Every strategy below has a true Sharpe ratio of exactly zero over 250 daily returns, and they share a market factor. The best one still looks tradeable. SR₀ is where the best is expected to land by luck alone.

best, annualised

1.90

participation ratio

3.8

SR₀ from NSR₀ from participation ratiobest-4-2024annualised Sharpe ratio of each zero-skill strategy

How long a track record has to be

Frequency matters even at the same annualised Sharpe ratio, because fewer observations per year carry less information, and fat tails matter more than frequency. Monthly returns with the moments of the HFR hedge fund index need almost five years to show that a Sharpe ratio of 2 beats 1, against under three for daily returns.

Figure: minimum track record length, years

How long a track record must be before an observed annualised Sharpe ratio is significantly above a benchmark at 95%. The outlined cell is the example in Bailey and López de Prado (2012).

observed SR ↓ / benchmark →0.00.51.01.52.02.5
0.511·····
1.02.7111····
1.51.212.7211···
2.00.691.222.7311··
2.50.440.691.222.7411·
3.00.310.440.691.232.7611

A dot means the observed Sharpe ratio does not exceed the benchmark, so no track record is long enough. Hover a cell to read it.

The trial count that matters

N should be the number of distinct configurations tried, including the ones abandoned early. Because strategy variants are correlated, replacing it with an effective count such as the eigenvalue participation ratio looks more careful, but when V is measured from the trials themselves it corrects for correlation twice. A shared factor moves every trial’s Sharpe ratio together and so already shrinks their spread, which puts the correlation inside √V before N is ever touched.

Running the batch in the first demo checks this. With the distinct count, the best of a zero-skill corpus clears SR₀ about half the time at any correlation, which is what an expected maximum should do. With the participation ratio, 73 to 85 percent of correlated null corpora clear it in the package’s reference simulation, so a bar built that way lets most pure noise through.

Validation

The package reproduces every numerical example printed in the three source papers, and each one is a test that runs on Python 3.10 to 3.13 across Linux, macOS and Windows.

examplepaperpackage
DSR, 100 trials, skew −3, kurtosis 10SR₀ ≈ 0.1132, DSR 0.90040.1132, 0.9004
Same strategy after 46 trials0.95050.9505
Normal returns after 88 trials0.95050.9505
PSR, monthly SR 0.458 over 24 months0.982 normal, 0.913 non-normal0.982, 0.913
MinTRL, SR 2 against 1, daily / weekly / monthly2.73 / 2.83 / 3.24 years2.73 / 2.83 / 3.24
MinTRL, monthly, HFR index moments4.99 years4.99
Trials allowed by five years of data4545

Monte Carlo tests check the rest. PSR crosses 0.95 on about 5% of zero-skill samples, and the expected maximum matches simulated maxima, overstating them by 2.5% at ten trials and under 1% from a hundred. The minimum track record length lands exactly where PSR reaches its confidence level.

What a DSR does not tell you

A DSR above 0.95 says the selection is unlikely to be explained by the search and the shape of the returns alone. It says nothing about transaction costs, capacity, look-ahead in the data, regime change or trials that were never recorded. Nor does a DSR below 0.95 prove the absence of skill, particularly on short samples.

Every formula also assumes serially independent returns. Positions held across several bars make returns autocorrelated, so the package provides an AR(1) effective sample size to pass in place of the raw observation count.

Reproducibility

DaruFinance / deflated-sharpe

Python · open source reference implementation

Minimal invocation

from deflated_sharpe import deflated_sharpe_ratio, to_period_sharpe, to_period_variance

result = deflated_sharpe_ratio(
    sharpe=to_period_sharpe(2.5, 250),        # annualised 2.5 on daily data
    n_obs=1250,                               # five years
    n_trials=100,                             # configurations tried
    var_trials=to_period_variance(0.5, 250),  # variance of their Sharpe ratios
    skew=-3, kurtosis=10,                     # raw kurtosis: normal = 3
)
result.dsr   # 0.9004, below the 95% bar

# or from the command line
# dsr deflate --sharpe 2.5 --annual --periods-per-year 250 --n-obs 1250 \
#     --trials 100 --trial-variance 0.5 --skew -3 --kurtosis 10

References

  1. [1]Bailey, D. H. & López de Prado, M. (2014). The Deflated Sharpe Ratio: correcting for selection bias, backtest overfitting and non-normality. Journal of Portfolio Management 40(5), 94–107.
  2. [2]Bailey, D. H. & López de Prado, M. (2012). The Sharpe Ratio Efficient Frontier. Journal of Risk 15(2), 3–44.
  3. [3]Bailey, D. H., Borwein, J. M., López de Prado, M. & Zhu, Q. J. (2014). Pseudo-mathematics and financial charlatanism: the effects of backtest overfitting on out-of-sample performance. Notices of the American Mathematical Society 61(5), 458–471.
  4. [4]Lo, A. W. (2002). The statistics of Sharpe ratios. Financial Analysts Journal 58(4), 36–52.