repository is now a presentation-ready development prototype

docs/project_charter.md

@@ -1,122 +1,110 @@
 # Utah Vehicle Health project charter
 
-## Working title
+## Prototype status
 
-**Utah Vehicle Health**
-
-*What millions of emissions inspections reveal about the next test across the
-state of Utah county feeds available in this dataset.*
+**Presentation-ready development prototype.** Every analytical finding and every
+dashboard value comes from the private, page-sampled 10,000-vehicle development
+cohort. These values are **sample results, not population estimates**. They must
+not be used to estimate statewide or county prevalence, create county rankings,
+or make decisions about an individual vehicle.
 
 ## Product promise
 
-Explain how vehicle age, type, location, and prior inspection history relate to
-the chance of passing the first attempt of the next emissions-inspection
-episode.
+Utah Vehicle Health explains how information available before an inspection
+relates to the first-attempt non-pass outcome of a returning vehicle's next
+inspection episode.
 
-The product measures **emissions-inspection outcomes**. It must not describe its
-score as a diagnosis of overall mechanical reliability, roadworthiness, safety,
-or legal compliance.
+The product measures an emissions-inspection outcome. It does not measure
+overall vehicle health, mechanical reliability, safety, roadworthiness, or
+legal compliance.
 
 ## Primary research question
 
 > Using only information available before an inspection episode begins, how
-> accurately and reliably can we estimate whether its first attempt will pass?
+> well can a calibrated logistic regression estimate whether a returning
+> vehicle's next episode will have a first-attempt non-pass outcome?
 
-An episode groups attempts for the same internal vehicle token when the gap
-between consecutive attempts is 30 days or less. The target is the first attempt
-of a new episode, not every rapid retest.
+An episode groups consecutive attempts for the same private vehicle token when
+the gap is 30 days or less. The target is the first attempt of a new episode,
+not a rapid retest. A vehicle is eligible only after at least one completed
+prior episode.
 
-## Primary target
+## Binary target
 
-- `0 — pass`: normalized `PASS` or `P`
-- `1 — non-pass`: normalized `FAIL`, `F`, `REJECT`, or `ABORT`
-- Blank, null, and unrecognized results are unlabeled. They remain in chronology
-  and coverage reporting but are excluded as supervised targets.
+- `0 - pass`: normalized `PASS` or `P`
+- `1 - non-pass`: normalized `FAIL`, `F`, `REJECT`, or `ABORT`
+- unlabeled: blank, null, and unrecognized results
 
-Because reject and abort can reflect process/readiness problems rather than
-mechanical failure, the project will also report:
+Reject and abort can reflect process or readiness conditions rather than a
+mechanical failure. The prototype therefore says **non-pass**, never “vehicle
+failure,” when referring to the combined target. Multiclass outcome modeling
+and fail-only sensitivity analysis are outside the finished prototype scope.
 
-- a four-class pass/fail/reject/abort analysis; and
-- a fail-versus-pass sensitivity analysis that excludes reject and abort.
+## Data and product boundary
 
-## Scope
+The development cohort contains complete inspection histories for up to 10,000
+sampled vehicles from participating Utah county/source feeds. Page sampling
+over-represents vehicles with more inspection records, so neither the cohort nor
+its dashboard aggregates are population-representative. The feeds do not cover
+all 29 Utah counties.
 
-The returning-vehicle MVP uses universal inspection-history fields and prior
-DMV information where it is point-in-time valid. It covers participating
-inspection county/source feeds, not all 29 Utah counties.
+The final model is inspection-history only. DMV enrichment, rich current-test
+OBD fields, station effects, cold-start prediction, and individualized lookup
+are outside scope. The static dashboard publishes only rounded, suppressed
+sample aggregates and pre-2025 development diagnostics.
 
-The rich OBD/odometer fields are a later, source-specific extension for newer
-`slco`, `davis`, and `cache` records. They are not part of the statewide-style
-historical baseline.
+## Final model and benchmark
 
-## Deliverables
+The final model is regularized logistic regression (`C=0.03`) with Platt
+probability calibration fit on the 2024 development-sample partition. A
+histogram gradient-boosted tree is retained only as a nonlinear benchmark. It
+is not a second final model and is not used to drive the product.
 
-1. A reproducible, read-only extraction and private pseudonymization pipeline.
-2. A point-in-time episode/feature mart with an auditable exclusion report.
-3. Prevalence, previous-outcome, and logistic-regression baselines.
-4. One calibrated boosted-tree model and an inspection-only versus DMV-enhanced
-   ablation.
-5. A locked temporal evaluation with subgroup and source-era diagnostics.
-6. Versioned, suppressed public aggregates for a four-page Bolt dashboard.
-7. A model card and data/methodology page documenting limitations.
+## Completed deliverables
 
-## Headline success criteria
+1. Read-only extraction with private vehicle pseudonymization.
+2. Leakage-safe episode and point-in-time feature mart.
+3. Training-prevalence and previous-episode baselines.
+4. Calibrated logistic regression as the final model.
+5. Histogram gradient boosting as a benchmark only.
+6. Chronological development evaluation with an explicit one-time 2025 gate.
+7. Suppression-reviewed static dashboard assets with fail-closed validation.
+8. Model card, final report, presentation materials, and dashboard-only Bolt
+   deployment instructions.
 
-- Beat both the training-prevalence and previous-episode-outcome baselines on
-  2025 PR-AUC and Brier score.
-- Produce calibrated probabilities, not just class labels.
-- Report performance by county/source era, vehicle-age band, history depth, and
-  DMV match/staleness.
-- Reproduce all published charts from versioned sanitized outputs.
-- Export no VIN, plate, ZIP, station, raw JSON, operational record, or
-  row-level prediction to Bolt.
-- Suppress public cells with fewer than 100 eligible inspections or fewer than
-  10 observations in an outcome or its complement, with complementary
-  suppression where totals could reveal a hidden cell.
+## Acceptance criteria
 
-## Initial feasibility result
-
-Under label contract v3, the fixed-seed aggregate-only query in
-`sql/10_episode_cohort_feasibility.sql` produced 8,916 eligible
-returning-vehicle episodes from 1,730 of 2,000 sampled vehicle histories. The
-first-attempt non-pass rate was 11.80%, and the median gap from the prior
-episode was about 372 days. Of those targets, 1,732 use the documented Utah OBD
-binary proxy because that source leaves its overall-result field blank. This
-supports both the episode definition and a calibrated binary model. The sample
-is for pipeline feasibility, not a population estimate.
-
-The 2025 sample showed a longer median gap and different outcome mix, reinforcing
-the need for source-era drift reporting and a locked chronological test.
-
-## Development holdout status
-
-During live verification, the explicit 2025 gate was opened once on the
-page-sampled development extract after both candidate specifications had been
-fixed from pre-2025 data. No test-informed model change was made. The frozen
-logistic model remains selected, and a later complete-data 2025 analysis will
-be treated as confirmatory rather than described as a pristine unseen test.
-The audit trail and results are recorded in
-[development_results.md](development_results.md).
+- The prediction unit remains a returning vehicle's next-episode first attempt.
+- Model features exist before that episode begins.
+- The calibrated logistic model beats both simple baselines on development-
+  sample PR-AUC and Brier score.
+- The boosted tree is presented only as a benchmark.
+- Every displayed value is labeled as a private-sample result and not a
+  population estimate.
+- County views communicate feed coverage and sample context; they do not claim
+  population rankings or causal county differences.
+- No VIN, plate, ZIP, station, technician identifier, raw JSON, credential,
+  operational record, private token, or row-level prediction reaches the
+  dashboard or presentation materials.
+- The dashboard contains no vehicle-level estimator, input form, or prediction
+  service.
 
 ## Non-goals
 
-- Diagnosing an individual vehicle
-- Certifying that a vehicle will pass
-- Ranking or accusing inspection stations
-- Identifying owners or accepting VIN/plate input
-- Making causal claims about county programs
-- Treating missing outcomes as passes
+- Predicting the result of a first-observed vehicle
+- Diagnosing, certifying, or guaranteeing an individual vehicle outcome
+- Ranking counties, programs, stations, technicians, owners, or vehicles
+- Making causal claims about geography, vehicle makes, or inspection programs
+- Accepting VIN, plate, address, station, or diagnostic inputs
+- Publishing population prevalence from the development sample
+- Shipping a production decision service or live database connection
+- Completing DMV enrichment, four-class modeling, or failure-to-pass journeys
 
-## Staged build
+## Evaluation status
 
-1. **Foundation:** episode definition, label normalization, exclusions, secure
-   extraction, and coverage checks.
-2. **Baseline:** inspection-history feature mart and transparent baselines.
-3. **Enrichment:** canonical vehicle dimension and point-in-time DMV features.
-4. **Modeling:** tree model, calibration, locked test, drift and subgroup audits.
-5. **Product:** sanitized aggregate export and Bolt MVP.
-6. **Stretch:** failure-to-pass journeys, multiclass probabilities, and a
-   clearly scoped rich-OBD model.
-
-See [modeling_protocol.md](modeling_protocol.md) and
-[dashboard_spec.md](dashboard_spec.md) for the detailed contracts.
+The explicit 2025 gate was opened once after model specifications were frozen
+from pre-2025 data. No 2025-informed model change was made. The 2025 values are
+therefore a one-time **development-sample holdout**, not a pristine future test
+and not population performance. See [development_results.md](development_results.md)
+for the audit trail and [modeling_protocol.md](modeling_protocol.md) for the
+frozen evaluation contract.