# Utah Vehicle Health project charter ## Working title **Utah Vehicle Health** *What millions of emissions inspections reveal about the next test across the state of Utah county feeds available in this dataset.* ## Product promise Explain how vehicle age, type, location, and prior inspection history relate to the chance of passing the first attempt of the next emissions-inspection episode. The product measures **emissions-inspection outcomes**. It must not describe its score as a diagnosis of overall mechanical reliability, roadworthiness, safety, or legal compliance. ## Primary research question > Using only information available before an inspection episode begins, how > accurately and reliably can we estimate whether its first attempt will pass? An episode groups attempts for the same internal vehicle token when the gap between consecutive attempts is 30 days or less. The target is the first attempt of a new episode, not every rapid retest. ## Primary target - `0 — pass`: normalized `PASS` or `P` - `1 — non-pass`: normalized `FAIL`, `F`, `REJECT`, or `ABORT` - Blank, null, and unrecognized results are unlabeled. They remain in chronology and coverage reporting but are excluded as supervised targets. Because reject and abort can reflect process/readiness problems rather than mechanical failure, the project will also report: - a four-class pass/fail/reject/abort analysis; and - a fail-versus-pass sensitivity analysis that excludes reject and abort. ## Scope The returning-vehicle MVP uses universal inspection-history fields and prior DMV information where it is point-in-time valid. It covers participating inspection county/source feeds, not all 29 Utah counties. The rich OBD/odometer fields are a later, source-specific extension for newer `slco`, `davis`, and `cache` records. They are not part of the statewide-style historical baseline. ## Deliverables 1. A reproducible, read-only extraction and private pseudonymization pipeline. 2. A point-in-time episode/feature mart with an auditable exclusion report. 3. Prevalence, previous-outcome, and logistic-regression baselines. 4. One calibrated boosted-tree model and an inspection-only versus DMV-enhanced ablation. 5. A locked temporal evaluation with subgroup and source-era diagnostics. 6. Versioned, suppressed public aggregates for a four-page Bolt dashboard. 7. A model card and data/methodology page documenting limitations. ## Headline success criteria - Beat both the training-prevalence and previous-episode-outcome baselines on 2025 PR-AUC and Brier score. - Produce calibrated probabilities, not just class labels. - Report performance by county/source era, vehicle-age band, history depth, and DMV match/staleness. - Reproduce all published charts from versioned sanitized outputs. - Export no VIN, plate, ZIP, station, raw JSON, operational record, or row-level prediction to Bolt. - Suppress public cells with fewer than 100 eligible inspections or fewer than 10 observations in an outcome or its complement, with complementary suppression where totals could reveal a hidden cell. ## Initial feasibility result Under label contract v3, the fixed-seed aggregate-only query in `sql/10_episode_cohort_feasibility.sql` produced 8,916 eligible returning-vehicle episodes from 1,730 of 2,000 sampled vehicle histories. The first-attempt non-pass rate was 11.80%, and the median gap from the prior episode was about 372 days. Of those targets, 1,732 use the documented Utah OBD binary proxy because that source leaves its overall-result field blank. This supports both the episode definition and a calibrated binary model. The sample is for pipeline feasibility, not a population estimate. The 2025 sample showed a longer median gap and different outcome mix, reinforcing the need for source-era drift reporting and a locked chronological test. ## Development holdout status During live verification, the explicit 2025 gate was opened once on the page-sampled development extract after both candidate specifications had been fixed from pre-2025 data. No test-informed model change was made. The frozen logistic model remains selected, and a later complete-data 2025 analysis will be treated as confirmatory rather than described as a pristine unseen test. The audit trail and results are recorded in [development_results.md](development_results.md). ## Non-goals - Diagnosing an individual vehicle - Certifying that a vehicle will pass - Ranking or accusing inspection stations - Identifying owners or accepting VIN/plate input - Making causal claims about county programs - Treating missing outcomes as passes ## Staged build 1. **Foundation:** episode definition, label normalization, exclusions, secure extraction, and coverage checks. 2. **Baseline:** inspection-history feature mart and transparent baselines. 3. **Enrichment:** canonical vehicle dimension and point-in-time DMV features. 4. **Modeling:** tree model, calibration, locked test, drift and subgroup audits. 5. **Product:** sanitized aggregate export and Bolt MVP. 6. **Stretch:** failure-to-pass journeys, multiclass probabilities, and a clearly scoped rich-OBD model. See [modeling_protocol.md](modeling_protocol.md) and [dashboard_spec.md](dashboard_spec.md) for the detailed contracts.