Fractional Differentiation: Stationarity Without Losing Memory

Fractional differentiation resolves a tension at the heart of quant feature engineering: models need stationary inputs, but the standard way of achieving stationarity — taking integer differences (returns) — erases almost all of the memory that made the series predictive in the first place. Fractional differencing, introduced to finance by Marcos López de Prado, lets you remove just enough of the trend to pass a stationarity test while keeping most of the long memory. It is one of the rare techniques that genuinely improves the feature/stationarity trade-off rather than merely re-labeling it.

The stationarity-versus-memory dilemma

Most ML and statistical methods assume the input distribution is stable over time. Raw price levels are emphatically non-stationary — they trend, drift, and wander, so their mean and variance change. The textbook fix is differencing: rt = pt - p_{t-1}, or in log space, log returns. This works for stationarity, but it is a sledgehammer.

The problem is that a price series is highly autocorrelated and carries information in its levels — how far it sits from a moving average, how persistent a trend has been. First differencing throws essentially all of that away: returns are close to memoryless. You have traded a non-stationary series rich in memory for a stationary series with almost none. Many models then struggle because the predictive structure left in returns is faint.

The key insight: integer differencing (d = 1) is overkill. Stationarity is often achievable with a fractional amount of differencing — d = 0.3, say — which removes far less memory.

Differencing as a fractional operator

Differencing can be written using the backshift operator B, where B pt = p{t-1}. First differencing is (1 - B) p_t. The trick is to allow non-integer d and expand (1 - B)^d as a binomial series:

\[ (1 - B)^d = \sum_{k=0}^{\infty} \binom{d}{k} (-B)^k \]

This produces an infinite sequence of weights applied to lagged values. The weights follow a simple recurrence:

\[ w0 = 1, \qquad wk = -w_{k-1} \cdot \frac{d - k + 1}{k} \]

For integer d = 1 the series terminates after two terms (1, -1) — ordinary differencing. For fractional d, the weights decay slowly and never quite reach zero, which is exactly why memory is preserved: the differenced value still depends on many past observations, not just the previous one.

import numpy as np

def frac_diff_weights(d, size):
    w = [1.0]
    for k in range(1, size):
        w.append(-w[-1] * (d - k + 1) / k)
    return np.array(w[::-1])   # reversed so most recent obs has weight 1

d	Interpretation	Memory retained	Stationary?
0	Original series	All	No
~0.3–0.5	Fractional diff	Most	Often yes
1	Returns (first diff)	Almost none	Yes
2	Second diff	None + noise	Yes (overkill)

Fixed-width window implementation

The infinite weight series is impractical, so in practice you truncate it once the weights fall below a threshold tau. This gives a fixed-width window (FFD) that is applied as a rolling dot product — and importantly, every output value uses the same number of weights, so the transform is consistent across the series.

import numpy as np
import pandas as pd

def frac_diff_ffd(series, d, tau=1e-4):
    # build weights until they decay below tau
    w = [1.0]
    k = 1
    while True:
        wk = -w[-1] * (d - k + 1) / k
        if abs(wk) < tau:
            break
        w.append(wk)
        k += 1
    w = np.array(w[::-1])
    width = len(w)

    out = pd.Series(index=series.index, dtype=float)
    vals = series.values
    for i in range(width - 1, len(vals)):
        window = vals[i - width + 1: i + 1]
        out.iloc[i] = np.dot(w, window)
    return out

Apply this to log prices, not raw prices, so the operator acts on a scale where proportional moves are comparable.

Finding the minimum d

The goal is the smallest d that makes the series stationary, because smaller d keeps more memory. The recipe:

1. Sweep d from 0 to 1 in small steps.
2. For each, compute the fractionally differenced series and run an **Augmented

Dickey-Fuller (ADF)** test.

1. Record the ADF statistic and the correlation between the differenced series and

the original log-price series (a proxy for retained memory).

1. Pick the smallest d whose ADF statistic crosses the chosen significance threshold.

from statsmodels.tsa.stattools import adfuller
import numpy as np

logp = np.log(price)
results = []
for d in np.linspace(0, 1, 21):
    ffd = frac_diff_ffd(logp, d).dropna()
    adf_stat, pval = adfuller(ffd, maxlag=1, regression="c", autolag=None)[:2]
    corr = np.corrcoef(logp.loc[ffd.index], ffd)[0, 1]
    results.append((round(d, 2), adf_stat, pval, corr))

res = pd.DataFrame(results, columns=["d", "adf_stat", "pval", "corr_to_orig"])
print(res)

Typically you find a sweet spot well below d = 1 — often in the 0.3–0.5 range for daily equity and FX prices — where the series is already stationary (ADF rejects the unit root) while correlation to the original log price remains high (say 0.8+). At that point you have a feature that is both model-friendly and memory-rich. Compare this with the near-zero correlation that full returns (d = 1) retain.

Where this helps in practice

Fractionally differenced prices make better inputs to ML models and statistical tests than raw returns when level information matters — for instance in cointegration-style relationships, distance from equilibrium, or persistent-trend features. They are a natural addition to a feature engineering pipeline for machine-learning trading, feeding tree models or networks a stationary series that still "remembers" where price has been.

Be realistic, though: fractional differencing is a better representation, not a source of alpha by itself. It removes a known defect (memoryless returns or non-stationary levels); it does not conjure predictability that the market does not offer.

Common pitfalls

• Defaulting to d = 1 out of habit and discarding nearly all memory.
• Differencing raw prices instead of log prices, mixing multiplicative and additive

scales.

• Forgetting to truncate weights, making the transform impossibly slow or

inconsistent across the series.

• Choosing d to maximize backtest performance rather than to minimally pass the ADF

test — that is overfitting the preprocessing.

• Recomputing weights with lookahead, e.g. fitting tau/d on the full sample then

applying it retroactively; choose d on in-sample data and lock it.

• Assuming stationarity is permanent — re-check d as regimes shift.
• Treating fractional diff as alpha instead of as a cleaner feature.

Key takeaways

• Integer differencing makes a series stationary

but destroys the memory that carried predictive information.

• Fractional differencing applies (1 - B)^d for non-integer d, with slowly-decaying

binomial weights that preserve long memory.

• Use a fixed-width window (truncate weights below tau) applied to log prices.
• Find the minimum d that passes the ADF test while keeping high correlation to the

original series — often 0.3–0.5 for daily data.

• It is a better input for ML models and

feature pipelines, not a standalone edge.

• Choose d without lookahead and revisit it as the market's behavior changes.