Real, usable edge for systematic traders — strategies, backtesting and code, no fluff

· 8 min read

Conditional Value at Risk (CVaR / Expected Shortfall) Explained

Conditional Value at Risk (CVaR) — also called expected shortfall (ES) — averages every loss beyond the VaR threshold, so it describes the shape of the tail rather than just its edge. It is a coherent risk measure, it rewards diversification, and it can be minimized with a linear program. But CVaR is also a tail-average estimator built from the few worst observations, which makes it noisier than VaR, and — unlike VaR — it is not elicitable, which makes it genuinely hard to backtest. This guide covers why VaR falls short, the precise definition of CVaR, historical and parametric estimation with their error properties, the Rockafellar-Uryasev optimization, and the honest limitations.

Why VaR is not enough

VaR at confidence level alpha is just a quantile of the loss distribution: the loss you exceed only (1 - alpha) of the time. It has two serious defects.

  • It is blind to tail shape. Two portfolios can share an identical 95% VaR of \$1M

while one loses \$1.1M in the worst 5% and the other loses \$20M. VaR cannot tell them apart, because it only looks at the boundary, not what lies beyond it.

  • It is not coherent. A risk measure is coherent if it satisfies four axioms:

monotonicity (more loss in every state means more risk), positive homogeneity (scaling the position scales the risk), translation invariance (adding cash reduces risk one-for-one), and — the one VaR violates — subadditivity: the risk of a combined book is never more than the sum of the parts. In plain notation, subadditivity requires rho(A + B) <= rho(A) + rho(B). VaR can break this with skewed or discrete payoffs (credit, options), producing the absurd result that splitting a book into two desks lowers measured risk.

CVaR fixes both. It is coherent (Artzner et al. 1999; Rockafellar-Uryasev 2000), and by construction it integrates the entire tail. Basel's Fundamental Review of the Trading Book moved bank capital from 99% VaR to 97.5% expected shortfall for exactly these reasons.

Defining CVaR

Let L be the loss (negative return) and alpha the confidence level, say 0.95. VaR is the alpha-quantile of losses:

VaR_alpha = inf{ L : P(loss <= L) >= alpha }

CVaR is the conditional expectation of loss given that loss is at least VaR:

CVaR_alpha = E[ L | L >= VaR_alpha ]

For continuous distributions this is exactly the average of the worst (1 - alpha) fraction of outcomes. CVaR is always at least as large as VaR, and the gap between them measures tail heaviness. A near-normal book has CVaR only slightly above VaR; a book stuffed with short options or tail-risk credit has CVaR far above its VaR — a warning VaR alone would never give you.

Estimation error: CVaR is noisier than VaR

This is the practical fact most treatments omit. CVaR averages the observations in the tail, and there are very few of them: at 95% with 250 daily returns, the tail is about 12 points; at 99% it is 2-3. Averaging 2-3 numbers gives a high-variance estimate, and because those points are themselves the most extreme draws, the CVaR estimator has both high variance and a downward small-sample bias (you have not yet observed the worst tail outcomes). The standard error of historical CVaR scales worse with alpha than VaR does. Two consequences:

  • Prefer 97.5% over 99% for empirical CVaR unless you have years of data.
  • Stabilize the tail with a parametric or extreme value theory

model rather than relying on a handful of empirical points.

Computing CVaR

Historical (empirical)

Sort realized returns, find the VaR quantile, average everything beyond it:

import numpy as np

def historical_var_cvar(returns, alpha=0.95):
    """returns: 1D array of period returns (losses are negative returns)."""
    losses = -np.asarray(returns)
    var = np.quantile(losses, alpha)            # VaR as a positive loss number
    tail = losses[losses >= var]                # the worst (1 - alpha) fraction
    cvar = tail.mean() if tail.size else var
    return var, cvar

r = np.random.default_rng(0).standard_t(4, 10_000) / 100   # fat-tailed sample
var, cvar = historical_var_cvar(r, 0.95)
print(f"95% VaR={var:.4%}  CVaR={cvar:.4%}")

The catch is the same as historical VaR, amplified: you can only ever see losses as large as the worst event already in your window, and the tail average is dominated by a few points. A calm sample yields a dangerously small CVaR.

Parametric (Gaussian and Student-t)

If you assume a distribution, CVaR has a closed form. For the normal distribution with mean mu and standard deviation sigma, writing phi for the standard normal density and z_alpha for the alpha-quantile of the standard normal:

CVaR_alpha = -mu + sigma * phi(z_alpha) / (1 - alpha)

from scipy.stats import norm, t

def gaussian_cvar(mu, sigma, alpha=0.95):
    z = norm.ppf(alpha)
    return -mu + sigma * norm.pdf(z) / (1 - alpha)

def student_t_cvar(mu, sigma, nu, alpha=0.95):
    """Fat-tailed CVaR for standardized Student-t with nu degrees of freedom."""
    x = t.ppf(alpha, nu)
    pdf = t.pdf(x, nu)
    factor = (nu + x**2) / (nu - 1)
    return -mu + sigma * pdf * factor / (1 - alpha)

print(gaussian_cvar(0.0, 0.015, 0.95))           # ~3.1% vs VaR ~2.5%
print(student_t_cvar(0.0, 0.015, nu=4, 0.95))    # materially larger

The Gaussian formula understates real tails. A Student-t with low degrees of freedom (fit nu on your data — 3-6 is typical for daily returns) gives a far more honest number and is barely more code.

MeasureWhat it reportsCoherent?Sees tail shape?
VolatilityDispersion (both sides)n/aNo
VaR_alphaThreshold lossNoNo
CVaR_alphaMean loss beyond VaRYesYes

CVaR portfolio optimization (Rockafellar-Uryasev)

The reason quants love CVaR is not just that it describes the tail — it is that minimizing CVaR is a convex (in fact linear) program, and you never need to compute VaR first. Rockafellar and Uryasev (2000) define an auxiliary function over portfolio weights w and a free scalar zeta (which converges to VaR at the optimum). Writing (x)+ for max(x, 0) and L(w) for the portfolio loss:

F_alpha(w, zeta) = zeta + (1 / (1 - alpha)) * E[ (L(w) - zeta)+ ]

Minimizing Falpha jointly over w and zeta minimizes CVaR. With T historical scenarios, the expectation becomes an average and the positive part is linearized with auxiliary variables ut >= 0, giving the linear program:

minimize over w, zeta, u:   zeta + (1 / ((1 - alpha) * T)) * sum_t u_t
subject to:                 u_t >= L_t(w) - zeta,   u_t >= 0,   sum(w) = 1

In Python with cvxpy:

import cvxpy as cp
import numpy as np

def min_cvar_portfolio(returns, alpha=0.95, target_ret=None):
    R = np.asarray(returns)            # shape (T scenarios, N assets)
    T, N = R.shape
    w = cp.Variable(N)
    zeta = cp.Variable()
    u = cp.Variable(T, nonneg=True)
    losses = -R @ w                    # scenario losses
    cvar = zeta + cp.sum(u) / ((1 - alpha) * T)

    cons = [u >= losses - zeta, cp.sum(w) == 1, w >= 0]
    if target_ret is not None:
        cons.append(R.mean(axis=0) @ w >= target_ret)

    cp.Problem(cp.Minimize(cvar), cons).solve()
    return w.value, cvar.value

w, cvar = min_cvar_portfolio(np.random.default_rng(1).normal(0, 0.01, (500, 8)))
print(w.round(3), f"CVaR={cvar:.4%}")

Because the whole thing is linear, it scales to thousands of scenarios and assets, handles non-normal and non-parametric loss distributions natively, and slots into the same constraint framework as mean-variance optimization — you simply swap variance for CVaR in the objective. You can also add CVaR as a constraint (keep CVaR below a budget while maximizing expected return), which is how risk desks often deploy it. The caveat: with few scenarios the in-sample CVaR-optimal weights overfit the empirical tail, so the out-of-sample CVaR is typically worse than reported.

The backtesting problem: ES is not elicitable

A subtle but important limitation: VaR is elicitable (there is a scoring function whose expected minimizer is the quantile), which is why simple breach-counting (Kupiec, Christoffersen) backtests it cleanly. CVaR is not elicitable on its own (Gneiting 2011). This makes direct backtesting harder; in practice you use joint VaR/ES backtests (Acerbi-Szekely 2014) or the fact that VaR and ES together are jointly elicitable. The takeaway for practitioners: do not assume ES is as easy to validate as VaR — it is not, and a poorly validated ES model can be confidently wrong.

CVaR vs other risk views

CVaR is a single-period, distribution-level measure. It does not capture the path of losses, which is where maximum drawdown matters: a strategy can have a modest one-day CVaR yet grind through a brutal multi-month drawdown. Use CVaR to size single-period tail risk and drawdown analysis to understand the path. Together with VaR and stress testing they form a layered picture rather than competing single numbers.

How CVaR is gamed and where it misleads

  • Confidence-level and convention shopping. Quoting 95% when 99% is unflattering,

or mixing loss conventions.

  • Optimizing in-sample. A CVaR-minimal portfolio fit and evaluated on the same data

overstates safety; validate out-of-sample and prefer regularized scenario sets.

  • Gaussian on fat tails. The normal formula understates the tail just like

parametric VaR — fit a Student-t or go historical with EVT.

  • Too few tail scenarios. At 95% with 200 samples, ~10 points define the tail and

~2 at 99%; the estimate is noisy and downward-biased. Use long samples or EVT.

  • Liquidity. CVaR assumes you exit at marked prices; in a crash, liquidation itself

moves the market against you, so realized tail losses exceed the model.

Key takeaways

  • CVaR (expected shortfall) averages losses beyond VaR, capturing the tail shape

that VaR ignores, and is coherent (subadditive), which is why Basel moved to 97.5% ES.

  • It is noisier than VaR because it averages the few worst observations; prefer

97.5% over 99% empirically and stabilize the tail with a Student-t or EVT model.

  • CVaR optimization is a linear program (Rockafellar-Uryasev) that drops straight

into cvxpy, but the in-sample optimum overfits the tail.

  • ES is not elicitable, so it is harder to backtest than VaR — validate with joint

VaR/ES procedures.

VaR, and a clear-eyed view of liquidity; no single number captures every way a book can lose.

#CVaR #expected shortfall #tail risk #value at risk #risk management

← All posts