Purged and Combinatorial Cross-Validation for Finance

Standard k-fold cross-validation is quietly broken for financial data, and it lies in your favor. In a typical machine learning for trading pipeline you shuffle samples into folds, train on some, validate on the rest, and trust the average score. That works when observations are independent. Financial labels are not: they overlap in time, returns are autocorrelated, and features are sticky. The result is leakage — information from the validation set bleeds into training — and validation scores that look far better than reality. This article explains why, then shows purging, the embargo, and combinatorial purged cross-validation (CPCV), the framework popularized by López de Prado, with Python sketches.

Why standard k-fold leaks

Two structural features of financial data break the independence assumption that ordinary k-fold relies on.

• Overlapping labels. A common labeling scheme assigns each observation a label based on what happens over the next h bars (a forward return, or a triple-barrier outcome). Two samples whose horizons overlap share information. If one lands in training and the temporally adjacent one in validation, the model has effectively seen the answer.
• Autocorrelation. Prices, volatility, and many engineered features are highly persistent. Adjacent observations are near-duplicates, so a randomly chosen validation point usually has an almost-identical neighbor in the training set.

Both effects mean the "out-of-sample" fold is contaminated by the training fold. The model gets credit for memorizing, not generalizing — a textbook recipe for overfitting that survives naive CV.

Purging

Purging removes from the training set any observation whose label horizon overlaps the validation set's time span. If a training sample's evaluation window touches any timestamp used by a validation sample, drop it. This severs the information bridge created by overlapping labels.

Concretely, each observation i has an interval [tstarti, tendi] over which its label is determined. When a block of observations is held out for validation, you purge every training observation whose interval intersects the validation block's interval.

The embargo

Purging handles direct overlap, but autocorrelation can leak across a sharp boundary even without overlapping label windows: the bar immediately after the validation block is correlated with the validation period. The embargo drops a small additional buffer of training observations immediately after each validation block (typically 1–5% of the sample). It is a one-sided gap that absorbs residual serial correlation.

Technique	What it removes	Fixes
Purging	Training samples whose label window overlaps the test set	Overlapping-label leakage
Embargo	A buffer of samples right after the test set	Autocorrelation leakage
Both	The union of the above	Most train/test contamination

Combinatorial purged cross-validation

Walk-forward and ordinary k-fold each give you a single out-of-sample path through history. That one path is itself a sample of one — noisy, and easy to get lucky on. CPCV generates many OOS paths by splitting the data into N groups and testing on all combinations of k groups at once, purging and embargoing each time. With N = 6 groups and k = 2 test groups, you get C(6,2) = 15 train/test splits and can reconstruct multiple full backtest paths.

The payoff: instead of one Sharpe or one accuracy number, you get a distribution of OOS results. That distribution is exactly what you need for the PBO calculation and for honest confidence intervals. CPCV is the rigorous cousin of walk-forward optimization: walk-forward respects time ordering sequentially, while CPCV trades strict ordering for many resampled paths under careful purging.

import numpy as np
import pandas as pd
from itertools import combinations

def purged_cv_splits(t_start, t_end, n_groups=6, k_test=2, embargo_pct=0.01):
    """Yield (train_idx, test_idx) for combinatorial purged CV.

    t_start, t_end: pandas Series of label start/end times, indexed 0..n-1.
    """
    n = len(t_start)
    groups = np.array_split(np.arange(n), n_groups)
    embargo = int(n * embargo_pct)

    for test_groups in combinations(range(n_groups), k_test):
        test_idx = np.concatenate([groups[g] for g in test_groups])
        test_idx.sort()
        test_t0, test_t1 = t_start.iloc[test_idx].min(), t_end.iloc[test_idx].max()

        train_idx = []
        for i in range(n):
            if i in set(test_idx):
                continue
            # PURGE: drop if label window overlaps the test window
            if t_end.iloc[i] >= test_t0 and t_start.iloc[i] <= test_t1:
                continue
            # EMBARGO: drop a buffer just after the test block
            if 0 <= (i - test_idx.max()) <= embargo:
                continue
            train_idx.append(i)
        yield np.array(train_idx), test_idx

In practice you would plug these splits into crossvalscore-style loops, collect the OOS metric for each split, and study the whole distribution rather than its mean alone.

Walk-forward vs CPCV: when to use which

Both methods give out-of-sample estimates, but they answer slightly different questions, and mature research pipelines use both.

• Walk-forward preserves strict chronological order: train on the past, test on the immediate future, roll forward. It mimics how the strategy would actually have been deployed and is the right final check before going live. Its weakness is that it produces a single historical path — one realization of a noisy process.
• CPCV sacrifices strict ordering (within careful purging) to manufacture many train/test combinations, yielding a distribution of OOS metrics. It is the better tool for estimating the variance of your edge and for feeding overfitting diagnostics.

A reasonable workflow is to develop and stress-test with CPCV to understand the spread of outcomes, then confirm the surviving configuration with a strict walk-forward pass that respects deployment ordering. If the walk-forward result lands inside the CPCV distribution, you have consistent evidence; if it falls in the tail, treat the edge as fragile.

Practical guidance

• Define honest label windows. Purging is only as good as your tstart/tend bookkeeping. Get the horizon right, including any look-ahead embedded in feature engineering.
• Size the embargo to the autocorrelation. Estimate how many bars of serial dependence your features carry and set the embargo at least that long.
• Use enough groups. More groups give more OOS paths but smaller test sets; N between 6 and 10 is a common balance.
• Report the distribution. The variance of OOS scores across paths is information — a wide spread warns that the edge is fragile.

Common mistakes

• Shuffling time-series data. Random shuffling destroys the temporal structure you need to purge and almost guarantees leakage.
• Forgetting feature look-ahead. A rolling feature that peeks forward (e.g. centered moving averages, full-sample scaling) leaks regardless of how carefully you purge labels.
• Skipping the embargo. Purging without an embargo still leaks through autocorrelation at fold boundaries.
• Reading only the mean OOS score. CPCV's value is the distribution; collapsing it to one number discards the robustness signal.
• Tiny test groups. If each test fold is too small, individual scores are too noisy to interpret.

Key takeaways

• Standard k-fold leaks in finance because labels overlap and features are autocorrelated.
• Purging removes training samples whose label windows overlap the test set.
• The embargo adds a buffer after each test block to kill residual autocorrelation.
• Combinatorial purged CV produces many OOS paths, yielding distributions instead of single noisy estimates.
• Honest label windows, look-ahead-free features, and distribution-level reporting are what make ML backtests trustworthy.