MLB Backtest Analysis
The same four-channel framework used elsewhere on this site, applied to MLB run-margin prediction and walk-forward tested — with the synthetic-data disclosure stated up front, not buried.
Three models, four channels
The framework splits MLB game features into channels — market, team strength, context, matchup — and compares three prediction approaches on run margin:
- Model 1 (Market baseline): moneyline odds only.
- Model 2 (Fundamental only): pitcher ERA, team strength, context, matchup — no market data.
- Model 3 (Residual correction): market baseline, then fundamentals correct the residual.
Results, independently recomputed
| Model | MAE (runs) | Improvement over Model 1 |
|---|---|---|
| Model 1 — Market | 1.581 | — |
| Model 2 — Fundamental | 1.259 | 20.4% |
| Model 3 — Residual correction | 1.303 | 17.5% |
Walk-forward tested across 12 sequential folds through the 2023–2024 window; Model 2 (fundamentals alone) won the majority of individual folds, not just the aggregate.
What this doesn't support
The source analysis is direct about this, and it's worth repeating rather than softening: prediction error of about ±1.3 runs doesn't clear the roughly 2-run edge needed to overcome standard vigorish betting margins directly. The honest application isn't "bet on this," it's calibrating fair-line estimates and comparing them against posted odds — and even that claim needs real market data, not synthetic data, before it means anything.
What would need to change to make this a real result
Stated directly in the source material rather than glossed over: real MLB history in place of synthetic data (the actual next step, not yet done), real scraped moneyline odds instead of synthetic market proxies, and accounting for what the synthetic data explicitly doesn't model — injury volatility, pitcher rest and workload, weather, and mid-season roster changes.
Comments