Utah Vehicle Health Returning vehicles · next first attempt

Utah Vehicle Health

Predicting a returning vehicle's next-episode first-attempt non-pass outcome.

One focused prediction question — a development prototype, not a population study or decision tool.

Kevin Bell · Summer 2026 development prototype

Research question

Using only information available before an inspection episode begins, how well can a calibrated logistic regression estimate whether a returning vehicle's next episode will have a first-attempt non-pass outcome?

Attempts ≤ 30 days apart form one episode — rapid fail/retest sequences never become repeated target rows.
The target is the first attempt of a returning vehicle's next episode.
Non-pass combines fail, reject, and abort; blanks stay unlabeled.

Development data

Complete histories for a non-representative sample

sample up to 10,000 vehicles ↓ retrieve their full histories ↓ build 30-day episodes ↓ 44,659 eligible returning targets

9,996private tokens after validation

83,552source events

69,588episodes

44,659eligible targets

Pipeline reconciliation counts — not Utah totals. Page sampling over-represents vehicles with more records, and feeds do not cover all 29 counties.

Leakage-safe design

Every feature window ends before the target

2010–15 context 2016–22 train 2023 tune 2024 calibrate 2025 one-time holdout 2026 partial shadow

Uses (all pre-episode)

Prior outcomes, history depth, timing
Vehicle age, season, public county context
Previously observed make / model

Excludes

Current-test diagnostics & later attempts
Future records, identifiers, stations, technicians

The 2025 gate was opened once after specs were frozen — a development holdout, not a pristine future test.

Model comparison

2025 one-time sample holdout · 3,971 non-audit episodes from 3,734 vehicles, 487 non-pass · higher PR-AUC and lower Brier are better · not population performance
Model	Role	PR-AUC	Brier
Training prevalence	Baseline	0.123	0.1076
Previous episode	Baseline	0.158	0.1731
Logistic + Platt	Final	0.261	0.1011
Boosted tree + Platt	Benchmark	0.238	0.1024

Ranked by score, the top 10% of episodes are non-pass 31.2% of the time against a 12.3% base rate — about 2.5× lift, roughly 1 in 3 instead of 1 in 8.

Logistic beat both baselines and the boosted-tree benchmark. We avoid the word “accuracy,” which hides class imbalance and calibration.

Model decision

Keep calibrated logistic regression

Final model: calibrated logistic regression (C=0.03, Platt scaling).
Benchmark only: histogram gradient boosting — not a second final model.
Selected on 2023 tune, where logistic led (0.282 vs 0.274 PR-AUC). 2024 only fits Platt scaling, so it cannot select; 2025 confirmed the choice.
The tree did not improve the declared sample metrics enough to justify its complexity.
Logistic regression is easier to audit and explain.

Calibration

The probabilities mean what they say

Each dot is one decile of the 2025 holdout — predicted probability across, observed non-pass rate up.
The dashed line is perfect calibration. Deciles rise monotonically and track it across the range.
Lowest decile: predicted 0.047, observed 0.040. Highest: predicted 0.348, observed 0.312.
This is what the Brier score of 0.1011 summarizes in a single number.

Mid-range bins wobble — each holds roughly 400 episodes, so a few outcomes move a point.

Live dashboard demo · 2:45

Switch to the dashboard

Follow demo_script.md. Keep the development-preview banner in view. Show logistic as final, the tree as benchmark-only, and that there is no vehicle-level prediction service.

1 · Overview 2 · Sample cohorts 3 · Model & benchmark 4 · Data & methods

node dashboard/server.mjs --lan → open the “Other devices” URL + /#overview

Privacy architecture

A deliberately narrow public boundary

private read-only data → local modeling → suppressed aggregate JSON → allowlisted local static server → same-network demo browser

No VINs, plates, ZIPs, stations, technician IDs, private tokens, raw JSON, credentials, operational records, or row-level predictions.
The browser has no database connection.
The dashboard fails closed if its aggregate contract does not validate.

Limitations

What this sample cannot claim

The page-sampled cohort is not population-representative.
Feeds are partial and change over time.
Geography and source era are confounded.
Reject / abort do not necessarily mean mechanical failure.
2025 is a one-time development holdout, not external validation.
Sample associations do not support county rankings or causal conclusions.

Close

A complete, honest development prototype

One precise question · a leakage-safe timeline · calibrated logistic regression as the final model · a boosted-tree benchmark · a privacy-reviewed static dashboard.

The value is the discipline of making only the claims the data and release boundary can support.

Questions?

Happy to go deep on episode construction, calibration, privacy, or the dashboard.