Measuring Volatility in Trading: Historical, Realized and Implied

Volatility is not one quantity but a family of estimators of an unobservable latent process, and the gap between them is where most risk-model error lives. Close- to-close standard deviation, range-based estimators, realized variance from intraday data, EWMA, GARCH, and option-implied vol all target "volatility" but make different assumptions and have wildly different efficiency and bias. Choosing the wrong one for the horizon — or annualizing it incorrectly — propagates straight into your VaR, leverage, and option marks. This article is about the estimators, their sampling properties, and the microstructure and look-ahead traps that corrupt them.

The estimand and the scaling law

Volatility is the square root of the variance of returns over an interval. Under the random-walk null, log returns are additive, so variance scales linearly with time and volatility scales with its square root: sigma(h) = sigma(1) sqrt(h). This is why daily vol annualizes as sigma_daily sqrt(252) and sqrt(252) ~= 15.87 is worth memorizing. But the square-root-of-time rule is exact only if returns are i.i.d. Two empirical facts break it:

• Autocorrelation. If returns are serially correlated (momentum or reversion),

multi-period variance is sum of variances + 2sum of autocovariances. Positive autocorrelation makes true multi-day vol higher* than the sqrt-time scaling implies; negative makes it lower. Scaling daily vol to annual blindly mis-states risk for any non-trivial strategy.

• Volatility clustering. Variance is itself time-varying, so the "constant

sigma" in the scaling law does not exist. The scaled number is an average over a regime, not a forecast.

Always use log returns for vol work — they are additive across time, making the scaling internally consistent. Simple returns are not, and the discrepancy compounds when aggregating.

import numpy as np
import pandas as pd

def close_to_close_vol(prices, window=20, ann=252):
    r = np.log(prices / prices.shift(1))
    return r.rolling(window).std(ddof=1) * np.sqrt(ann)

Estimator efficiency: why range beats close-to-close

Close-to-close vol uses one data point per day and is statistically inefficient — it discards the path the price took intraday. Range-based estimators (Parkinson, Garman-Klass, Rogers-Satchell, Yang-Zhang) use the high, low, open, and close, and are dramatically more efficient: Garman-Klass is roughly 7-8x more efficient than close-to-close, meaning you get the same estimator variance from far fewer days.

def garman_klass(df, window=20, ann=252):
    # df has columns: Open, High, Low, Close
    log_hl = np.log(df["High"] / df["Low"])
    log_co = np.log(df["Close"] / df["Open"])
    rs = 0.5 * log_hl**2 - (2 * np.log(2) - 1) * log_co**2
    return np.sqrt(rs.rolling(window).mean() * ann)

def yang_zhang(df, window=20, ann=252):
    # Handles opening jumps and drift; most robust of the range estimators
    o = np.log(df["Open"] / df["Close"].shift(1))
    c = np.log(df["Close"] / df["Open"])
    rs = (np.log(df["High"]/df["Open"]) * np.log(df["High"]/df["Close"]) +
          np.log(df["Low"]/df["Open"])  * np.log(df["Low"]/df["Close"]))
    k = 0.34 / (1.34 + (window + 1) / (window - 1))
    vo = o.rolling(window).var(ddof=1)
    vc = c.rolling(window).var(ddof=1)
    vrs = rs.rolling(window).mean()
    return np.sqrt((vo + k * vc + (1 - k) * vrs) * ann)

The caveats: Parkinson and Garman-Klass assume no drift and no opening gap, so they understate vol for assets that jump at the open (most equities). Yang-Zhang is the robust default because it is drift-independent and incorporates overnight returns. All range estimators assume continuous observation of the true high/low, which discrete sampling biases downward — the observed range is always inside the true range.

Realized variance and microstructure noise

With intraday data you estimate a day's variation by summing squared high-frequency returns: RV = sum of r_i^2 over the day. As sampling frequency increases, RV converges to the integrated variance — in theory. In practice it does not, because of microstructure noise: bid-ask bounce, discrete prices, and asynchronous quotes inject a noise term whose variance accumulates with the number of samples. Sample every tick and RV explodes; the signature plot of RV versus sampling frequency diverges at high frequency.

The standard mitigations: sample at a moderate frequency (5 minutes is the canonical compromise), use a noise-robust estimator like the two-scale realized variance or a realized kernel, and add an overnight-gap term since intraday RV misses the close- to-open move that often carries the day's largest jump. Realized vol is the most responsive estimator and the right input for intraday risk and volatility trading, but only once the noise is handled.

EWMA and GARCH: smoothing versus structure

A rectangular window weights all observations equally and then drops them discontinuously, causing "ghosting" — a large return inflates the estimate for exactly window days, then vanishes. EWMA fixes the abruptness:

sigma2_t = lam * sigma2_{t-1} + (1 - lam) * r_{t-1}^2

with lam = 0.94 for daily data (RiskMetrics). This is a constrained IGARCH: it has no mean-reversion term, so its multi-step forecast is flat — fine for one-day risk, wrong for term structure. GARCH(1,1) adds a constant and a true persistence term, sigma2t = omega + alphar{t-1}^2 + betasigma2_{t-1}, giving mean-reversion toward omega/(1-alpha-beta) and a forecast that decays from the current level. EWMA is a fast, fit-free baseline; GARCH is the forecasting tool when you need a term structure of variance.

Estimator	Data needed	Strength	Main weakness
Close-to-close	daily OHLC	trivial, transparent	inefficient, ghosting
Garman-Klass / Yang-Zhang	daily OHLC	5-8x efficiency	assumes/needs gaps handled
Realized variance	intraday	most responsive, model-free	microstructure noise
EWMA	daily returns	smooth, no fit	flat forecast, fixed decay
GARCH	daily returns	mean-reverting forecast	estimation risk, refit drift
Implied	option chain	forward-looking	embeds risk premium, model-dependent

Implied volatility and the risk premium

Implied vol is the volatility input that equates a model price (e.g. Black-Scholes) to the market option price — the market's risk-neutral expectation of future vol. It is the only forward- looking estimator, but it is not a clean forecast of realized vol: it embeds the variance risk premium, the compensation sellers demand for bearing variance risk. Implied systematically exceeds subsequent realized vol on average, which is precisely the edge harvested by selling options and trading the vol risk premium.

Two structural features: implied vol varies by strike (the smile/skew), so there is no single number — equity index puts trade at elevated implied vol because crash protection is persistently bid. And implied vol is model-dependent; quoting it requires agreeing on the pricing model. Comparing a constant-strike implied to realized requires care because the option's moneyness drifts as spot moves.

Volatility drives sizing — and the look-ahead trap

Position size scales inversely with volatility so each position contributes comparable risk; this is the basis of volatility targeting and risk parity.

def vol_target_weight(target_ann_vol, forecast_ann_vol, vol_floor=0.05):
    # Floor prevents leverage blow-up in ultra-calm regimes
    return target_ann_vol / max(forecast_ann_vol, vol_floor)

The subtle, expensive error here is look-ahead. The vol estimate that sizes today's position must use only data available at the close before the position is taken. A rolling std aligned to the same bar it scales, or a centered window, leaks future information and makes a vol-targeted backtest look far smoother than it traded. Equally, vol targeting reduces exposure into rising vol, which empirically improves drawdown control and risk-adjusted returns, but it does so at the cost of turnover and whipsaw — the estimator's responsiveness is a direct cost driver, not free.

Jumps, bipower variation, and Jensen bias

Two refinements separate professional volatility measurement from the textbook version. First, realized variance lumps together continuous diffusion and discrete jumps, but they have different risk and forecasting properties — jump risk is largely undiversifiable and not well-hedged by vol targeting. Bipower variation (the sum of products of adjacent absolute returns) estimates the continuous part only, so the difference RV - BV isolates the jump contribution. Splitting them lets you forecast the persistent continuous component separately from the unpredictable jump component, which improves out-of-sample variance forecasts.

import numpy as np

def jump_decomposition(intraday_log_returns):
    r = np.asarray(intraday_log_returns)
    rv = np.sum(r**2)                                  # realized variance
    mu1 = np.sqrt(2 / np.pi)
    bv = (1 / mu1**2) * np.sum(np.abs(r[1:]) * np.abs(r[:-1]))  # bipower variation
    jump = max(rv - bv, 0.0)                           # jump contribution (truncated)
    return dict(rv=rv, continuous=bv, jump=jump)

Second, a quiet bias: practitioners often forecast variance and then take the square root to quote volatility, but by Jensen's inequality E[sqrt(var)] <= sqrt(E[var]). Averaging variances and rooting is not the same as averaging volatilities; the gap grows with the dispersion of the variance estimates. When aggregating or forecasting, be explicit about whether you are operating in variance space (additive, the natural space for the math) or volatility space (what you quote), and convert once at the end.

Honest limits

Every volatility number is an estimate of a latent process under assumptions that financial data violates: i.i.d. returns (broken by autocorrelation and clustering), continuous prices (broken by jumps and microstructure), and stable regimes (broken by crises). The sqrt-time scaling is an approximation, range estimators assume away gaps, realized vol fights noise, GARCH carries estimation risk and must be refit, and implied vol is contaminated by a risk premium and a model choice. The professional posture is to triangulate — use a fast estimator for live risk control, a structural model for the forecast term structure, realized vol for intraday, and implied for the forward-looking, risk-premium-bearing view — and to floor your estimates and quote your conventions (returns type, periods-per-year) explicitly. Measure it several ways, update it often, and never treat a single volatility number as the truth about a quantity you cannot directly observe.