Authority Handoff Prediction — Tesla Autopilot

A real, five-state model predicting when human-machine authority transfer will fail — before it fails — built on real NTSB crash investigation data, not synthetic examples. Not aimed at any one company; aimed at anyone who'd recognize rigorous work on this problem.

The real, honest scope — stated directly, not smoothed over

The original target, from the project's own planning document, before any work began: 100 NTSB cases for model fitting, 50 held-out cases for forward-walk validation. What actually got built: 27 cases for fitting, 14 for forward-walk — 41 total. The reduction is real, and the reason is stated directly rather than hidden: extracting and annotating each real NTSB case to the standard this methodology requires took far more effort per case than expected. This is published as a demonstration that the method works, not as a claim that 41 cases proves a production-ready system. A full-scale version, at the original 100+50 target, is expected to perform considerably better than a statistics-only approach — stated as an expectation here, not claimed as proven.

The core model

The dangerous moment in a human-machine system isn't the collision. It's the earlier moment when authority transfer becomes structurally impossible. Five states, mapped directly onto real channel-divergence metrics:

Driven by S_vs_D (environment diverging from the system's model), D_vs_I (system-human readiness mismatch), I_vs_C (human-execution gap), and S_vs_C — the same channel-divergence structure used across every other domain mapping on this site, applied here to a real safety-critical case.

Real forward-walk discipline

The 14 held-out test cases were not researched, selected, or extracted until after the model's weights were completely fixed on the 27 training cases — the correct way to avoid look-ahead bias. The downloadable test-set file has every ground-truth field explicitly set to null and is marked "status": "BLIND" in the file itself, not just described that way afterward.

Real results

71.4% on genuinely unseen cases, against a 42.3% baseline from the Phase 1B backtest — a real +29.1 percentage point improvement. TACC is the honest weak point: the light-level gate was firing before the TACC object-detection gate on several cases, a specific, diagnosed failure mode, not a vague miss.

A real, self-identified risk — stated rather than hidden

Ground truth for the forward-walk cases was partly inferred using the same gate logic the model uses to predict, creating a real possibility of circular validation on the cases with the strongest signals. This is named directly in the downloadable results, with an honest likelihood estimate attached rather than dismissed — the same standard applied to every other finding on this site.

Real case data

Annotations reference real NTSB investigation numbers — including HWY18FH011 (Mountain View, CA, March 2018) and HWY16FH018 (Williston, FL) — both real, publicly documented fatal Tesla Autopilot crash investigations, not constructed examples.

Generalization beyond Tesla — named as a claim to test, not asserted as proven

The same five-state progression is mapped, in the full methodology, onto aviation autopilot, medical AI decision support, and industrial robotics — stated explicitly as the next thing to validate, not something already validated here.

A further comparison — what it actually is, and isn't

A follow-up analysis compares a single-signal detector (flagging risk only from sustained steering-wheel-torque absence) against a structured, four-channel risk score, applied to the same real NTSB cases above. Stated directly, because it matters: this is not a benchmark against Tesla's actual internal system. We don't have Tesla's real sensor stream — cameras, radar, and internal confidence scores sampled continuously — and we don't have their real detection algorithm. What's available publicly is NTSB's retrospective investigative summary: a small number of hand-selected narrative datapoints written to document what happened, not raw telemetry. What this comparison actually tests is two different analysis methods, applied after the fact to the same limited public record — a real, legitimate, but much narrower thing than comparing against Tesla's real system.

The real result: the torque-only detector flagged risk in all 16 available cases — fatal and non-fatal alike — which is a real, demonstrable problem with a single-signal approach: constant firing has no actual discriminative value, even when it happens to be "right" on the cases that mattered. A first attempt at a structured score, built from a narrow proxy signal, only flagged 1 of 16. A second attempt, built around the literal documented physical failure pattern in the richest available case (a lead-vehicle "cut-out" followed by TACC accelerating into the gap), flagged 0 of 16 — worse on the headline number, and honestly reported as worse, not adjusted until it looked better.

The honest diagnosis: 11 of the 16 real cases have only 2 recorded datapoints, which structurally isn't enough temporal resolution for any real pattern — including the real, physically correct one — to register, regardless of how the scoring formula is written. The real bottleneck here is data richness in the public record, not the four-channel approach itself. This is included on the site as a real, honestly-reported result, not a validated finding — the responsible next step is either richer source data or a properly fitted threshold on a larger real case set, not further hand-tuning of constants until a result looks better.

Get the whole thing