Gaussian Processes for Trading Signals
Gaussian processes (GPs) are Bayesian models over functions. Rather than committing to one fitted curve, a GP produces a posterior distribution for the unknown mapping from features to a return, probability, volatility, or cost target. That uncertainty is valuable in markets: a forecast far from training data should usually receive less capital, not the same confidence as an interpolated one.
The price is scalability and kernel risk. Exact GP inference costs cubically in the number of training observations, and a flexible kernel can manufacture smooth-looking patterns from weak evidence. A GP is most useful for small-to-medium, well-curated datasets or as a benchmark that exposes whether a more complex model is honestly uncertain.
Prior, kernel, posterior
Model a latent function as f(x) ~ GP(m(x), k(x,x')), then observe y = f(x) + epsilon. The kernel encodes similarity. Given training inputs X, posterior mean and variance at x* are:
mean = k_*' (K + sigma_n^2 I)^-1 y
var = k(x*,x*) - k_*' (K + sigma_n^2 I)^-1 k_*
The posterior variance is conditional on the specified kernel and parameters; it is not a full guarantee against structural breaks or data errors.
| Kernel | Assumption | Finance use |
|---|---|---|
| RBF | very smooth function | smooth cost curve, usually too smooth for alpha |
| Matérn | controlled roughness | cross-sectional signal mapping |
| Periodic | repeating cycle | calendar effect only with strong evidence |
| Additive | separate effects | value + momentum + liquidity |
| Product | interactions | regime-dependent response |
Use a mean function appropriate to the target. For excess returns, zero or a benchmark forecast may be sensible; for spreads or volatility, a transformed target and nonzero baseline often make more sense.
A chronological regression example
The example uses an expanding training window and a Matérn kernel. Feature scaling belongs inside the training pipeline, so its parameters never see future observations.
from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import Matern, WhiteKernel, ConstantKernel
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
kernel = ConstantKernel(1.0, (1e-3, 1e3)) * Matern(nu=1.5) + WhiteKernel()
gp = make_pipeline(
StandardScaler(),
GaussianProcessRegressor(kernel=kernel, normalize_y=True,
n_restarts_optimizer=3, random_state=7)
)
gp.fit(X_train_asof, y_train_asof)
mu, std = gp.predict(X_next, return_std=True)
score = mu / np.maximum(std, 1e-6)
mu / std is a ranking heuristic, not a universal position-size formula. Predictive variance can be low because the model is confidently wrong. Clip exposure, combine with forecast-residual risk, and monitor calibration after each retrain.
Kernel engineering is hypothesis engineering
For time-series signals, an RBF kernel over a timestamp implies that returns change smoothly through calendar time—usually implausible. It may fit a trend that will not repeat. Instead, construct economically motivated, causal features such as valuation, carry, volatility, and liquidity, then use a kernel over those features.
Additive kernels support attribution: one component can model the smooth response to value, another the rougher response to momentum. Product kernels allow interactions but enlarge the search. Predeclare a limited kernel family and evaluate it in nested walk-forward validation, as with any machine learning trading model.
| Result | Possible diagnosis | Next action |
|---|---|---|
| high mean, high uncertainty | extrapolation | reduce size or collect data |
| high train likelihood, weak OOS | kernel overfit | simplify kernel |
| residual autocorrelation | omitted time dynamics | add causal lags or state |
| poor interval coverage | misspecified noise | heteroskedastic likelihood |
Scaling beyond exact inference
Exact GPs become impractical as N grows. Sparse variational GPs summarize data through inducing points; structured kernel interpolation exploits grids; local GPs partition the feature space. These approximations add choices that affect uncertainty, so compare their calibration and runtime against an exact GP on a representative subset.
For noisy financial targets, consider Student-t likelihoods, heteroskedastic models, or a GP over log variance rather than assuming constant Gaussian residuals. Classification GPs can model directional probabilities, but their approximate inference should be checked with reliability curves and Brier scores.
From forecast to a portfolio
Evaluate both predictive and economic performance. A good log predictive density with no net P&L may be useful for risk or execution rather than alpha. Convert forecasts to a constrained portfolio with neutralization, liquidity limits, borrow assumptions, and cost estimates. Compare an uncertainty-aware GP allocation to a mean-only version: if uncertainty does not improve drawdowns, turnover, or calibration out of sample, it may not justify complexity.
Treat retraining cadence as an experiment. Refitting hyperparameters daily can chase noise and create unstable scores. Log learned length scales, noise levels, forecast dispersion, and feature-distance diagnostics so a regime shift is visible before it becomes unexplained P&L.
Key takeaways
- Gaussian processes produce a forecast distribution, making uncertainty actionable for sizing and abstention.
- A kernel expresses a market hypothesis and must be selected with time-respecting validation.
- Exact inference is expensive; sparse approximations require their own calibration checks.
- Compare net portfolio results and forecast coverage against simple, robust baselines.