Shadow Trading: Closing the Paper-to-Live Gap

Shadow trading (also called parallel paper trading) runs your strategy's signals and simulated fills alongside live markets — or alongside a small live sleeve — to measure the gap between research assumptions and reality. Unlike pre-launch paper trading, shadowing continues after go-live as a permanent diagnostic. This article covers what to log, how to attribute the paper-live gap, and promotion criteria that keep broken strategies from scaling.

Why backtests and live diverge

Source of gap	What happens
Fill model	Mid fills vs bid/ask / queue
Latency	Signal time ≠ order time
Partial fills	Live clips; sim assumes full
Adverse selection	Live fills when you are wrong
Data differences	Research feed ≠ production feed
Code path drift	Research notebook ≠ production binary
Costs	Fees, borrow, funding omitted or wrong
Capacity	Live size moves the market

Shadow trading isolates these by holding signals fixed and comparing execution layers.

Architectures

1. Signal twin (recommended baseline)

Production computes signals once; two execution adapters consume them:

Live router → real orders
Shadow simulator → hypothetical fills with the same timestamps

2. Full paper twin

Separate process reads the same market data, computes signals independently, simulates fills. Catches signal bugs and data skew — higher ops cost.

3. Tiny live sleeve + full shadow

Trade 1–5% of target size live; shadow the full size. Compare scaled P&L and slippage.

live_pnl_scaled = live_pnl / live_fraction
gap = shadow_pnl - live_pnl_scaled

What to log every decision

Minimum schema:

record = {
    "ts_signal": "...",
    "ts_order": "...",
    "symbol": "BTCUSDT",
    "side": "buy",
    "qty_intended": 1.0,
    "qty_live": 0.7,
    "px_signal_mid": 65000.0,
    "px_live_avg": 65012.0,
    "px_shadow": 65005.0,
    "fee_live": 0.5,
    "reject_reason": None,
}

Without tssignal vs tsorder, you cannot separate latency from slippage.

Decomposing the gap

gap ≈ alpha_model_gap + timing_gap + fill_gap + cost_gap + size_gap

Component	Diagnostic
Alpha model	Shadow with perfect mid fills still ≠ research backtest → data/code drift
Timing	Delay signal→order; markout over delay window
Fill	Live avg vs shadow aggressive/passive model
Cost	Fees/funding/borrow difference
Size	Gap grows with size → impact

def gap_report(live_rets, shadow_rets, research_rets) -> dict:
    return {
        "live_vs_shadow": (live_rets - shadow_rets).mean(),
        "shadow_vs_research": (shadow_rets - research_rets).mean(),
        "live_vs_research": (live_rets - research_rets).mean(),
    }

If shadow ≈ research but live ≪ shadow, the bug is execution. If shadow ≪ research, the bug is simulation fidelity or data.

Fill models for the shadow book

Match your intent:

Marketable — fill at ask/bid + fee; optional adverse selection haircut
Passive — queue model or fill only if trade-through; low fill rate is realistic
Auction — separate logic for open/close

Optimistic mid fills make shadow trading useless — they hide the gap you need to see. Use the same conservatism as event-driven backtests.

Promotion gates

Do not scale until:

Shadow and research agree within X bps/day for N weeks
Live sleeve tracks shadow within Y bps after costs
Kill-switches tested (live deployment)
TCA markouts stable
No unexplained rejects / data gaps

If live underperforms shadow only on high-vol days, fix toxicity handling before scaling (adverse selection).

Organizational habit

Shadow trading is an ops practice:

Dashboard: live vs shadow P&L, turnover, fill rate
Alert if \|gap\| exceeds threshold for D days
Freeze research deployments that break shadow parity
Quarterly audit: research code hash vs production hash

The goal is not perfect equality — markets are noisy — but explained differences.

Key takeaways

Shadow trading compares research, simulation, and live on the same signals
Log signal time, order time, intended qty, and fill prices
Decompose gaps into model, timing, fill, cost, and size
Use conservative shadow fills — mid-price shadows hide problems
Scale only after live sleeve tracks shadow under explicit gates