repository is now a presentation-ready development prototype
This commit is contained in:
+101
-72
@@ -1,97 +1,126 @@
|
||||
# Bolt dashboard specification
|
||||
# Static dashboard specification
|
||||
|
||||
## Product language
|
||||
## Release status and language
|
||||
|
||||
Use **Utah Vehicle Health** as the brand, but call the modeled quantity
|
||||
**next-episode non-pass risk**. The estimator is a cohort estimate, not a
|
||||
diagnosis, certification, or guarantee.
|
||||
The checked-in site is a **development preview** built from suppressed
|
||||
aggregates derived from the private, page-sampled 10,000-vehicle development
|
||||
cohort. Every value is a sample result, **not a population estimate**. The site
|
||||
must not present statewide prevalence, county rankings, causal comparisons, or
|
||||
individual predictions.
|
||||
|
||||
## Four-page MVP
|
||||
Use **Utah Vehicle Health** as the project name and **next-episode first-attempt
|
||||
non-pass** for the modeled outcome. Non-pass combines fail, reject, and abort.
|
||||
The dashboard is not a diagnosis, certification, safety assessment,
|
||||
roadworthiness assessment, or guarantee.
|
||||
|
||||
The sections below describe the full product target. The checked-in
|
||||
development preview intentionally implements a narrower safe subset: binary
|
||||
pass/non-pass aggregates, supported make/model scorecards, age bands, coverage
|
||||
quality, and pre-2025 model diagnostics. Four-class charts, uncertainty
|
||||
intervals, adjusted scorecards, and the prediction lookup remain disabled until
|
||||
their own reviewed aggregate assets exist.
|
||||
## Implemented four-view prototype
|
||||
|
||||
### Overview
|
||||
|
||||
- Eligible inspections, pass rate, non-pass rate, and covered-period KPIs
|
||||
- Quarterly pass/fail/reject/abort trend with blanks shown separately
|
||||
- Covered-county map; unavailable counties remain gray
|
||||
- Non-pass risk versus vehicle age with intervals
|
||||
- Clear notices for partial periods and limited feed coverage
|
||||
- Visibly persistent development-preview warning
|
||||
- Rounded published support and sample pass/non-pass rates
|
||||
- Sample first-attempt non-pass trend for approved, pre-2025 periods
|
||||
- County **feed-coverage** map, with unavailable counties shown as unavailable
|
||||
rather than zero
|
||||
- Sample non-pass pattern by coarsened vehicle-age band
|
||||
- Plain-language target and limitations callout
|
||||
|
||||
### Reliability explorer
|
||||
County values provide sample/feed context only. They must not be sorted,
|
||||
headlined, or described as population county performance.
|
||||
|
||||
- Search supported canonical make/model cohorts
|
||||
- Compare up to three cohorts across vehicle-age bands
|
||||
- Observed versus model-adjusted risk toggle
|
||||
- Outcome-mix bar and uncertainty-aware ranked dot plot
|
||||
- County, make/model, age band, fuel, program, and period filters
|
||||
- Support size and interval displayed for every estimate
|
||||
### Sample cohorts
|
||||
|
||||
### Next-test risk estimator
|
||||
- Search and compare only supported, suppression-cleared make/model cohorts
|
||||
- Show observed sample non-pass rates and rounded support
|
||||
- Default sorting by support rather than risk
|
||||
- Disable filters or adjusted views that the aggregate bundle cannot support
|
||||
- Explain that omitted cohorts may be suppressed or unavailable
|
||||
|
||||
Inputs are coarsened, non-identifying attributes: county, supported make/model,
|
||||
vehicle-age band, fuel, prior episode outcome, time-since-prior band, season,
|
||||
and approved program category.
|
||||
These are descriptive development-sample cohorts, not reliability ratings,
|
||||
causal make/model effects, or recommendations.
|
||||
|
||||
Output a calibrated non-pass probability, uncertainty interval, relevant
|
||||
baseline, and aggregate factor contributions. Never request VIN, plate, exact
|
||||
address, station, free text, or current-test diagnostics.
|
||||
### Model & benchmark
|
||||
|
||||
### Data and methods
|
||||
- Identify calibrated logistic regression as the sole final prototype model.
|
||||
- Identify calibrated histogram gradient boosting as benchmark-only.
|
||||
- Show only pre-2025, 2024 calibration-fit diagnostics from the private sample.
|
||||
- Explain that those same-partition values are calibration checks, not
|
||||
independent final-performance or population estimates.
|
||||
- Keep the opened 2025 holdout comparison in the private report/presentation,
|
||||
not in the browser bundle.
|
||||
- State plainly that the finished prototype has no vehicle-level prediction
|
||||
service, personal input form, or row-level output.
|
||||
|
||||
- Coverage timeline and source/year missingness heatmap
|
||||
- Episode and target definitions
|
||||
- `slc`/`slco` source-era explanation
|
||||
- Temporal split, PR-AUC, Brier score, and calibration plot
|
||||
- Subgroup/source-era performance
|
||||
- Leakage controls, DMV gaps, partial periods, and limitations
|
||||
### Data & methods
|
||||
|
||||
- First-attempt returning-vehicle episode and binary-target definitions
|
||||
- Point-in-time feature and leakage controls
|
||||
- Coverage timeline by source era
|
||||
- Chronological development-sample split and one-time 2025-gate disclosure
|
||||
- Evidence-governance summary for the logistic final model and tree benchmark
|
||||
- Known sampling, coverage, label, validation, and interpretation limits
|
||||
- Private-to-public publication flow
|
||||
|
||||
The boosted tree appears only in benchmark context, never as a second final
|
||||
model. The dashboard never requests a VIN, plate, ZIP, exact address, station,
|
||||
technician, free text, or current-test diagnostic.
|
||||
|
||||
## Public data contract
|
||||
|
||||
| Dataset | Safe grain |
|
||||
| Asset | Allowed content |
|
||||
| --- | --- |
|
||||
| `data_manifest` | Data cutoff, deterministic release ID, model versions, definitions and exclusions |
|
||||
| `overview_period_county` | Quarter/year × public county with rounded support and outcome rates |
|
||||
| `cohort_scorecard` | Approved make/model × age band, optionally coarsened county/fuel |
|
||||
| `age_risk_curve` | Approved cohort × age point/band with risk, interval and support |
|
||||
| `prediction_lookup` | Only supported coarsened input combinations and calibrated outputs |
|
||||
| `filter_catalog` | Publishable categories and valid combinations |
|
||||
| `model_diagnostics` | Approved partition-level metrics; locked metrics require a separate release gate |
|
||||
| `coverage_quality` | Source era × year volume, blank rate, linkage and availability |
|
||||
| `data_manifest.json` | Release ID, model versions, scope flags, definitions, and suppression settings |
|
||||
| `overview_period_county.json` | Suppressed quarter x covered-feed county sample aggregates |
|
||||
| `cohort_scorecard.json` | Suppressed make/model sample aggregates with rounded support |
|
||||
| `age_risk_curve.json` | Suppressed sample rates by coarsened vehicle-age band |
|
||||
| `filter_catalog.json` | Publishable categories and supported periods |
|
||||
| `model_diagnostics.json` | Train/tune/calibration development metrics only; no 2025 metrics |
|
||||
| `coverage_quality.json` | Source-era availability by year |
|
||||
| `sha256_manifest.json` | Checksums for the approved bundle |
|
||||
|
||||
Use purpose-built outputs rather than a single high-dimensional browser cube.
|
||||
Bolt receives only these sanitized, versioned assets—never countydata
|
||||
credentials or private analytical rows.
|
||||
Every JSON asset must declare `development_preview=true` and
|
||||
`population_estimate_allowed=false`. The public manifest must declare
|
||||
`locked_test_metrics_published=false`. The browser validates the complete bundle
|
||||
and shows no estimates if any required asset fails its contract.
|
||||
|
||||
## Publication controls
|
||||
The browser bundle must never contain raw analytical rows, prediction lookup
|
||||
rows, hidden unsuppressed chart layers, or values recoverable only through the
|
||||
developer console.
|
||||
|
||||
- Suppress cells below 100 eligible inspections.
|
||||
- Also require at least 100 distinct private vehicle tokens in every published
|
||||
cell; tokens and distinct counts never enter the public asset.
|
||||
- Suppress when an outcome or its complement has fewer than 10 records.
|
||||
- Require at least 10 distinct vehicles contributing each binary class.
|
||||
- Apply complementary suppression so totals cannot reconstruct hidden cells.
|
||||
- Combine rare categories, coarsen model years, and round displayed counts.
|
||||
- Recheck thresholds after every filter combination.
|
||||
- Do not include suppressed rows in browser bundles, API responses, downloads,
|
||||
analytics logs, or hidden chart layers.
|
||||
- Downloads contain only the sanitized summary currently displayed.
|
||||
## Suppression and privacy controls
|
||||
|
||||
## Visual direction
|
||||
- Require at least 100 eligible episodes and 100 distinct private vehicles per
|
||||
published cell.
|
||||
- Require at least 10 observations and 10 distinct vehicles in both binary
|
||||
outcome classes.
|
||||
- Apply complementary suppression when a visible total could reveal a hidden
|
||||
cell.
|
||||
- Combine rare categories, coarsen age/year fields, and round support.
|
||||
- Recheck thresholds for the exact grain of every released asset.
|
||||
- Omit suppressed rows entirely; do not send them to the browser.
|
||||
- Publish no VIN, plate, ZIP, exact address, station, technician identifier,
|
||||
private token, raw JSON, credential, operational record, free text, or
|
||||
row-level prediction.
|
||||
- Make no browser or hosted-service connection to `countydata`.
|
||||
|
||||
Use a restrained Utah/desert palette: teal pass, red fail, amber reject, purple
|
||||
abort, and gray missing. Use probability bars, calibrated dot plots, confidence
|
||||
bands, and cohort comparisons instead of gauges. Do not rely on color alone.
|
||||
## Accessibility and visual behavior
|
||||
|
||||
## Stretch pages
|
||||
- Use semantic headings, landmarks, labels, status messages, and keyboard-
|
||||
reachable navigation.
|
||||
- Provide text alternatives or accessible labels for charts.
|
||||
- Do not rely on color alone.
|
||||
- Use a restrained Utah/desert palette and direct probability/rate encodings;
|
||||
avoid gauges and decorative risk scores.
|
||||
- Preserve the warning banner and fail-closed unavailable state on desktop and
|
||||
mobile layouts.
|
||||
|
||||
- Failure-to-pass journeys with funnels, attempts-to-pass and survival curves
|
||||
- Four-class outcome probabilities
|
||||
- Source-scoped OBD early-warning analysis
|
||||
- Automated aggregate refresh with a dedicated read-only role
|
||||
- Model-drift monitoring
|
||||
## Deployment boundary
|
||||
|
||||
Bolt receives a separate dashboard-only project whose root is the contents of
|
||||
`dashboard/`. Do not import or upload the full private-pipeline repository and
|
||||
do not rely on a configurable subdirectory working root. The deployable project
|
||||
contains no `.env`, private `data/`, model artifacts, SQL, pipeline scripts, or
|
||||
repository history outside the dashboard directory.
|
||||
|
||||
See [bolt_deployment.md](bolt_deployment.md) for the reviewed handoff and
|
||||
post-publication checks.
|
||||
|
||||
Reference in New Issue
Block a user