Three changes to the presentation, all sourced from existing artifacts:
Slide 5 (model comparison): caption now states the evaluation set
(3,971 non-audit episodes from 3,734 vehicles, 487 non-pass), and a
callout reports top-10% precision of 31.2% against the 12.3% base
rate — about 2.5x lift. The two headline metrics alone gave the
audience no way to judge whether 0.261 PR-AUC is good.
Slide 6 (model decision): record that candidate selection happened on
the 2023 tune partition, where logistic led 0.282 to 0.274, and that
2024 fits Platt scaling only and so cannot select. The dashboard demo
displays 2024 numbers where the tree leads on both metrics; without
this line the deck has no answer to the obvious question.
New slide 7 (calibration): reliability plot of the 2025 holdout
deciles against the perfect-calibration diagonal, plotted from
artifacts/private/holdout/baseline_v1/calibration_bins.csv. Nothing in
the deck or dashboard previously showed calibration, though it is the
centerpiece claim of the final model.
Values verified against artifacts/private/holdout/*/metrics.json and
calibration_bins.csv. Deck remains self-contained; print/PDF backup
exports 11 pages.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
kevinbellandClaude Opus 5