Feature Engineering for Trading: Building Predictive Inputs

In machine-learning trading, feature engineering is where most of the realized edge — and almost all of the realized risk — actually lives. With a near-zero signal-to-noise ratio, the marginal return on a better algorithm is small while the marginal return on a feature that captures real market structure is large. The job is to turn economic priors into stationary, leak-free numbers, prove they carry incremental information, and discard the rest. This guide treats feature engineering as a measurement problem with strict point-in-time discipline, not a catalog of indicators.

Four non-negotiable principles

• Stationarity. Models need inputs whose distribution is stable over time. Use

returns, ratios, and rolling z-scores, not raw prices. See stationarity in time series.

• No leakage. Every feature at time t uses only information available at or

before t. Lag by one bar and scale with backward-looking statistics only.

• Economic rationale. Prefer features with a mechanism (risk premium, flow,

friction) over arbitrary transforms. Mechanisms decay slower than data-mined quirks.

• Incremental information. A feature earns inclusion only if it adds

out-of-sample predictive power given the features you already have — not in isolation.

Stationarity without destroying memory

Differencing (returns) makes a series stationary but erases nearly all memory, which is wasteful when the level itself carries slow-moving information. Fractional differentiation removes just enough memory to pass an ADF test while preserving predictive structure — often a better feature than plain returns for level-aware signals. See fractional differentiation and confirm stationarity formally rather than by eye.

Categories of features

Price / momentum

Multi-horizon returns (1, 5, 20, 60, 120 bars) let the model see the shape of recent action; z-scored distance from a moving average answers "how stretched is price?" — a natural input for mean-reversion and trend models.

A blunt warning on classic technical indicators. RSI, MACD, and Bollinger %B are weak features: they are deterministic, near-collinear transforms of the same lagged price path, they are computed by everyone, and the trivial edges they once held are arbitraged. In practice they fail the tests that matter — low and unstable information coefficient, fast decay, and high mutual correlation with plain momentum features that already subsume them. If you include them at all, treat them as redundant candidates to be pruned by correlation and permutation tests, never as recommended inputs.

Volatility

Rolling realized volatility, Parkinson/Garman-Klass range estimators, and volatility-of-volatility. These are predictive on their own (vol clusters and mean-reverts — see measuring volatility) and, crucially, act as conditioning variables: most edges, momentum especially, behave differently across volatility regimes.

Volume / liquidity / microstructure

Volume z-score, dollar volume, Amihud illiquidity, and — if you have book data — order-flow imbalance and signed volume. These tend to carry the most genuine short-horizon signal because they measure the mechanics of price formation rather than its shadow.

Cross-sectional

Rank an asset's return, value, or carry against its universe at each point in time. Cross-sectional ranks strip the market move, are far more stationary than raw levels, and are directly tradeable as a long-top/short-bottom portfolio — the foundation of factor investing. Cross-sectional features have a second, underrated virtue: they are self-normalizing. A rank in [0, 1] or a cross-sectional z-score has the same distribution every day regardless of the market's absolute level or volatility, so the feature never drifts out of the range the model trained on. This is the cleanest practical answer to non-stationarity, which is why equity-factor desks lean on it so heavily.

Interaction and conditioning features

Because most market relationships are conditional, an explicit interaction term often adds more than a new raw feature. Momentum times a low-volatility indicator, carry times a liquidity rank, or a signal times a regime flag lets a linear model express "this edge only works here" without a tree. Build a small number of economically-justified interactions by hand rather than letting an automated feature-cross explode the dimensionality — the latter is a fast route to overfitting on a small effective sample.

A worked, leak-aware feature block

Everything below uses only past data through shift, rolling windows that end at the current bar, and a final lag so the feature aligned with bar t predates the decision.

import numpy as np
import pandas as pd

def make_features(df: pd.DataFrame) -> pd.DataFrame:
    out = pd.DataFrame(index=df.index)
    logp = np.log(df["close"])
    ret = logp.diff()

    # Multi-horizon momentum, volatility-normalized (stationary, scale-free)
    rvol = ret.rolling(20).std()
    for h in (5, 20, 60, 120):
        out[f"mom_{h}"] = ret.rolling(h).sum() / (rvol * np.sqrt(h))

    # Stretch: z-scored distance from a moving average
    ma = df["close"].rolling(20).mean()
    dist = (df["close"] - ma) / ma
    out["stretch_z"] = (dist - dist.rolling(120).mean()) / dist.rolling(120).std()

    # Realized vol level and vol-of-vol
    out["rvol_20"] = rvol * np.sqrt(252)
    out["vol_of_vol"] = rvol.rolling(60).std()

    # Liquidity: Amihud illiquidity and volume surprise
    dollar_vol = df["close"] * df["volume"]
    out["amihud"] = (ret.abs() / dollar_vol).rolling(20).mean()
    v = df["volume"]
    out["vol_z"] = (v - v.rolling(60).mean()) / v.rolling(60).std()

    # Regime conditioner: is recent vol above its trailing median?
    out["high_vol"] = (out["rvol_20"] >
                       out["rvol_20"].rolling(252).median()).astype(int)

    return out.shift(1)  # decision at bar t uses features through t-1

That final shift(1) is the single most important line: it guarantees no same-bar leakage. Note that every transform uses rolling statistics — none peek at the full sample.

Normalization without leakage

Scaling helps most models, but the scaler is itself a feature that can leak.

• Rolling/expanding z-scores keep features stationary and look only backward.
• Cross-sectional ranks per timestamp are robust to outliers and regime drift.
• Never fit a StandardScaler (or PCA, or target encoder) on the full series and

apply it across a train/test boundary — that injects the test distribution into training. Fit on train, transform forward, or compute statistics in a rolling window.

The rule generalizes: any statistic used to transform a feature must itself be computable from data strictly before the decision point.

Leakage traps and fixes

Trap	What goes wrong	Fix
Same-bar close	Feature uses the bar it trades on	Lag features by one bar (shift(1))
Global scaling / PCA	Stats include the test period	Rolling/expanding, or fit on train only
Target leakage	Feature derived from the label	Audit every feature's derivation
Back-fill	Missing values filled from the future	Forward-fill only
Restated/adjusted data	Hindsight splits, dividends, restatements	Point-in-time data
Survivorship universe	Delisted names excluded	Include dead tickers

Leakage is insidious because it only makes the backtest better. Trace each feature to the exact timestamps of the data it consumes; if a single one postdates the decision, the result is fiction.

Selecting features under a small effective sample

More features means more chances to fit noise, and with a small effective sample that risk compounds. Pruning is mandatory, not optional.

import numpy as np, pandas as pd
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.inspection import permutation_importance

def prune_features(X, y, corr_thresh=0.9):
    # 1) drop near-duplicate features by absolute correlation
    corr = X.corr().abs()
    upper = corr.where(np.triu(np.ones(corr.shape), k=1).astype(bool))
    drop = [c for c in upper.columns if (upper[c] > corr_thresh).any()]
    Xk = X.drop(columns=drop)

    # 2) keep only features with positive OOS permutation importance
    split = int(len(Xk) * 0.7)
    m = HistGradientBoostingClassifier(max_depth=3, learning_rate=0.05,
                                       l2_regularization=1.0).fit(
        Xk.iloc[:split], y.iloc[:split])
    pi = permutation_importance(m, Xk.iloc[split:], y.iloc[split:],
                                n_repeats=20, random_state=0)
    keep = Xk.columns[pi.importances_mean > 0]
    return list(keep), drop

Prefer permutation importance on held-out data over impurity-based importances, which inflate for correlated and high-cardinality inputs. Better still, evaluate a feature by the change in net out-of-sample Sharpe when it is added, and keep a written economic rationale for everything that survives. Pair this with regularization (L1 drives weak weights to zero) and with the discipline that nothing enters the model unless you can explain why it should predict.

Information decay and the speed of a feature

Every feature has a horizon over which its predictive content lives, and matching that horizon to your holding period is part of engineering it. A feature whose information coefficient peaks at a one-day forward return and is gone by five days is useless to a weekly strategy, and ruinous if you trade it weekly anyway — you pay costs to chase a signal that has already evaporated. Measure the IC of each feature against forward returns at several horizons and plot the decay curve; the shape tells you both the right holding period and whether the feature is a fast, high-turnover microstructure signal or a slow, low-turnover risk premium. Fast signals demand cheap execution and tight transaction-cost control; slow ones tolerate friction but need more breadth to be a business.

import pandas as pd

def ic_decay(feature: pd.Series, returns: pd.Series, horizons=(1, 5, 10, 20)):
    out = {}
    for h in horizons:
        fwd = returns.shift(-1).rolling(h).sum().shift(-(h - 1))  # forward h-bar return
        df = pd.concat([feature, fwd], axis=1).dropna()
        out[h] = df.iloc[:, 0].corr(df.iloc[:, 1], method="spearman")
    return pd.Series(out, name="IC")   # inspect where IC peaks and how fast it decays

Feature stability across regimes

A feature that predicts in one regime and is meaningless in another will confuse a model unless you also supply the context to distinguish regimes. Test each feature's IC across sub-periods and volatility buckets; a feature that only works in trends belongs behind a regime flag (or interacted with one), not in the model unconditionally. This is why volatility and regime conditioners often unlock the predictive power of everything else. When you have many correlated raw features, PCA can compress them into a few orthogonal inputs — but fit it on a rolling, in-sample window to avoid leakage.

Realistic expectations

A single engineered feature with a stable cross-sectional IC of 0.02-0.04 is a good feature. Most candidates you try will have an IC indistinguishable from zero out-of-sample; that is normal. The edge comes from combining several weak, lowly correlated features so their idiosyncratic noise cancels, then validating the combination — not from one magic transform. Judge the final feature set by net, cost-aware Sharpe under walk-forward testing, discounted for the number of features and configs you tried.

Conclusion

Feature engineering is where domain knowledge becomes alpha and where leakage quietly destroys it. Build stationary, economically-motivated, leak-free features; scale only with backward-looking statistics; and prune aggressively with correlation and out-of-sample permutation tests. Treat RSI, MACD, and Bollinger-style indicators as weak, collinear, easily-arbitraged candidates that usually fail IC and decay tests — not as recommended inputs. Then validate the whole pipeline with walk-forward analysis and judge it on net, cost-adjusted returns. Pair these features with the overfitting-resistant tree models that tend to use them best.