SummerProject2026/docs/project_charter.md

Utah Vehicle Health project charter

Prototype status

Presentation-ready development prototype. Every analytical finding and every dashboard value comes from the private, page-sampled 10,000-vehicle development cohort. These values are sample results, not population estimates. They must not be used to estimate statewide or county prevalence, create county rankings, or make decisions about an individual vehicle.

Product promise

Utah Vehicle Health explains how information available before an inspection relates to the first-attempt non-pass outcome of a returning vehicle's next inspection episode.

The product measures an emissions-inspection outcome. It does not measure overall vehicle health, mechanical reliability, safety, roadworthiness, or legal compliance.

Primary research question

Using only information available before an inspection episode begins, how well can a calibrated logistic regression estimate whether a returning vehicle's next episode will have a first-attempt non-pass outcome?

An episode groups consecutive attempts for the same private vehicle token when the gap is 30 days or less. The target is the first attempt of a new episode, not a rapid retest. A vehicle is eligible only after at least one completed prior episode.

Binary target

• 0 - pass: normalized PASS or P
• 1 - non-pass: normalized FAIL, F, REJECT, or ABORT
• unlabeled: blank, null, and unrecognized results

Reject and abort can reflect process or readiness conditions rather than a mechanical failure. The prototype therefore says non-pass, never “vehicle failure,” when referring to the combined target. Multiclass outcome modeling and fail-only sensitivity analysis are outside the finished prototype scope.

Data and product boundary

The development cohort contains complete inspection histories for up to 10,000 sampled vehicles from participating Utah county/source feeds. Page sampling over-represents vehicles with more inspection records, so neither the cohort nor its dashboard aggregates are population-representative. The feeds do not cover all 29 Utah counties.

The final model is inspection-history only. DMV enrichment, rich current-test OBD fields, station effects, cold-start prediction, and individualized lookup are outside scope. The static dashboard publishes only rounded, suppressed sample aggregates and pre-2025 development diagnostics.

Final model and benchmark

The final model is regularized logistic regression (C=0.03) with Platt probability calibration fit on the 2024 development-sample partition. A histogram gradient-boosted tree is retained only as a nonlinear benchmark. It is not a second final model and is not used to drive the product.

Completed deliverables

1. Read-only extraction with private vehicle pseudonymization.
2. Leakage-safe episode and point-in-time feature mart.
3. Training-prevalence and previous-episode baselines.
4. Calibrated logistic regression as the final model.
5. Histogram gradient boosting as a benchmark only.
6. Chronological development evaluation with an explicit one-time 2025 gate.
7. Suppression-reviewed static dashboard assets with fail-closed validation.
8. Model card, final report, presentation materials, and same-network dashboard demo instructions.

Acceptance criteria

• The prediction unit remains a returning vehicle's next-episode first attempt.
• Model features exist before that episode begins.
• The calibrated logistic model beats both simple baselines on development- sample PR-AUC and Brier score.
• The boosted tree is presented only as a benchmark.
• Every displayed value is labeled as a private-sample result and not a population estimate.
• County views communicate feed coverage and sample context; they do not claim population rankings or causal county differences.
• No VIN, plate, ZIP, station, technician identifier, raw JSON, credential, operational record, private token, or row-level prediction reaches the dashboard or presentation materials.
• The dashboard contains no vehicle-level estimator, input form, or prediction service.

Non-goals

• Predicting the result of a first-observed vehicle
• Diagnosing, certifying, or guaranteeing an individual vehicle outcome
• Ranking counties, programs, stations, technicians, owners, or vehicles
• Making causal claims about geography, vehicle makes, or inspection programs
• Accepting VIN, plate, address, station, or diagnostic inputs
• Publishing population prevalence from the development sample
• Shipping a production decision service or live database connection
• Completing DMV enrichment, four-class modeling, or failure-to-pass journeys

Evaluation status

The explicit 2025 gate was opened once after model specifications were frozen from pre-2025 data. No 2025-informed model change was made. The 2025 values are therefore a one-time development-sample holdout, not a pristine future test and not population performance. See development_results.md for the audit trail and modeling_protocol.md for the frozen evaluation contract.