Compare commits

...
18 Commits
Author SHA1 Message Date
kevinbell c6b36fa235 Merge pull request 'Add calibration slide and sharpen model-comparison evidence in deck' (#2) from deck-calibration-slide into main
Reviewed-on: #2
2026-08-27 23:00:44 +00:00
kevinbellandClaude Opus 5 180d58e646 Add calibration slide and sharpen model-comparison evidence in deck
Three changes to the presentation, all sourced from existing artifacts:

Slide 5 (model comparison): caption now states the evaluation set
(3,971 non-audit episodes from 3,734 vehicles, 487 non-pass), and a
callout reports top-10% precision of 31.2% against the 12.3% base
rate — about 2.5x lift. The two headline metrics alone gave the
audience no way to judge whether 0.261 PR-AUC is good.

Slide 6 (model decision): record that candidate selection happened on
the 2023 tune partition, where logistic led 0.282 to 0.274, and that
2024 fits Platt scaling only and so cannot select. The dashboard demo
displays 2024 numbers where the tree leads on both metrics; without
this line the deck has no answer to the obvious question.

New slide 7 (calibration): reliability plot of the 2025 holdout
deciles against the perfect-calibration diagonal, plotted from
artifacts/private/holdout/baseline_v1/calibration_bins.csv. Nothing in
the deck or dashboard previously showed calibration, though it is the
centerpiece claim of the final model.

Values verified against artifacts/private/holdout/*/metrics.json and
calibration_bins.csv. Deck remains self-contained; print/PDF backup
exports 11 pages.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 16:58:38 -06:00
kevinbell fbaf874f8d minor fixes 2026-08-27 16:47:09 -06:00
kevinbell 70bb5a4e07 Update README.md 2026-08-10 17:09:25 +00:00
kevinbell 27a483a86e Update README.md 2026-08-04 21:12:32 +00:00
kevinbell 98d0d24318 Update docs/architecture.md 2026-08-04 20:47:27 +00:00
kevinbell af987eb9e8 Update docs/architecture.md 2026-08-04 20:46:51 +00:00
kevinbell cbc80cf903 Update docs/architecture.mmd 2026-08-04 20:46:10 +00:00
kevinbell a3f64d44d9 added the macOS caffeinate command 2026-08-03 10:42:01 -06:00
kevinbell 70c37bc1b6 Update README.md 2026-08-03 16:37:34 +00:00
kevinbell f3b1843321 Update README.md 2026-07-31 18:39:38 +00:00
kevinbell 3ca57be97c website access minor edit 2026-07-29 11:43:18 -06:00
kevinbell d703f19ccc website access 2026-07-29 11:38:15 -06:00
kevinbell f2ab7f55f1 added presentation slides 2026-07-28 13:05:07 -06:00
kevinbell 7a9f4b6bc7 Update README.md 2026-07-28 17:23:31 +00:00
kevinbell fa2435388a clicking a tab no longer auto-scrolls 2026-07-27 14:44:17 -06:00
kevinbell bc9a133ee6 all public dashboard stuff changed to static dashboard stuff 2026-07-27 14:39:50 -06:00
kevinbell 786b896619 all public dashboard stuff changed to static dashboard stuff 2026-07-27 14:39:46 -06:00
10 changed files with 1379 additions and 215 deletions
+12 -10
View File
@@ -1,3 +1,5 @@
<a name="readme-top"></a>
# Utah Vehicle Health
Kevin Bell's summer 2026 development prototype for the first-attempt outcome of
@@ -28,10 +30,9 @@ Start with the [final report](docs/final_report.md). Supporting deliverables:
- [Detailed development results](docs/development_results.md)
- [Model card](docs/model_card.md)
- [Static dashboard specification](docs/dashboard_spec.md)
- [Private-to-public architecture](docs/architecture.mmd)
- [Private-to-public architecture](docs/architecture.md)
- [10-minute presentation outline](docs/presentation_outline.md)
- [Dashboard demo script](docs/demo_script.md)
- [Optional public-hosting notes](docs/bolt_deployment.md)
The [data inventory](docs/data_inventory.md) and archived
[project options](docs/project_options.md) provide source-discovery history;
@@ -63,7 +64,7 @@ python3 -m venv .venv
The local `.env` contains PostgreSQL connection values and the project-specific
`VIN_HASH_KEY`; it is excluded from Git. Never put those values in source code,
screenshots, browser JavaScript, presentation materials, or a Bolt project.
screenshots, browser JavaScript, or presentation materials.
## Safe database access in VS Code
@@ -151,6 +152,12 @@ repository root. Stop any older dashboard process with `Ctrl-C` first:
node dashboard/server.mjs --lan
```
On macOS, to prevent the host from sleeping while the demo runs:
```bash
caffeinate -i node dashboard/server.mjs --lan
```
The server prints a **This computer** URL and one or more interface-labeled
**Other devices** URLs. Open the URL for the shared Wi-Fi or Ethernet interface
on the second computer; do not use `0.0.0.0` as the browser address. Confirm the
@@ -163,13 +170,6 @@ page, so the dashboard includes a dependency-free checksum implementation for
that case. These checks validate bundle consistency; plain HTTP does not
authenticate the network transport, so use a trusted demo network.
## Public hosting is not required
The current presentation plan is the same-network live demo above. Nothing
needs to be uploaded or published, and the server exposes only the dashboard's
explicit static-file allowlist. The older
[public-hosting checklist](docs/bolt_deployment.md) is retained only in case the
delivery requirements change later.
## Author
@@ -178,3 +178,5 @@ Kevin Bell
<kevinBell@Linux.com>
<https://kevinbell.dev/>
<p align="left"><a href="#readme-top">back to top</a></p>
+6 -2
View File
@@ -41,6 +41,10 @@ address, not the address to put in a browser. Confirm the in-app status reads
**Validated sample aggregates**, then stop the server with `Ctrl-C` after the
demo.
If the host is on a Tailscale tailnet, the server also prints a **Tailscale
(tunnel)** URL (a `100.x.y.z` address). Open that URL on any other device signed
into the same tailnet to reach the dashboard remotely, without sharing a LAN.
All data-contract and checksum validation remains enabled over LAN HTTP. When a
remote browser does not expose the SubtleCrypto digest API to the plain-HTTP
page, the dashboard uses its dependency-free SHA-256 implementation and still
@@ -92,7 +96,7 @@ records must never enter this directory.
## Public hosting is not required
The current delivery path is the same-network live demo. No Bolt project,
public URL, dependency install, or build step is needed. If public hosting is
The delivery path is the same-network live demo. No public URL, dependency
install, or build step is needed. If public hosting is
ever reconsidered, publish only a separately reviewed copy of this dashboard
directory—never the private source repository.
+1 -1
View File
@@ -86,8 +86,8 @@ function showRoute({ announce = true, focus = false } = {}) {
if (heading) {
heading.setAttribute("tabindex", "-1");
heading.focus({ preventScroll: true });
heading.scrollIntoView({ behavior: "smooth", block: "start" });
}
window.scrollTo({ top: 0, left: 0, behavior: "instant" });
}
}
+7 -7
View File
@@ -52,12 +52,13 @@ function safeFile(requestUrl) {
}
}
// Tailscale and similar VPN tunnels show up as extra "LAN" interfaces but are not
// useful for an in-room demo, so keep them out of the printed URL list.
function isTunnelInterface(interfaceName, address) {
if (/^(utun|tailscale|tun|ppp|ipsec)/i.test(interfaceName)) return true;
// A tailnet advertises the host over a 100.64.0.0/10 address, so label that URL
// clearly for remote (off-LAN) demos over the tunnel.
function interfaceLabel(interfaceName, address) {
const [first, second] = address.split(".").map(Number);
return first === 100 && second >= 64 && second <= 127; // 100.64.0.0/10 (CGNAT/tailnet)
const isTailnet =
/tailscale/i.test(interfaceName) || (first === 100 && second >= 64 && second <= 127);
return isTailnet ? "Remote (tunnel)" : `Other devices (${interfaceName})`;
}
function displayUrls(host, port) {
@@ -72,10 +73,9 @@ function displayUrls(host, port) {
for (const [interfaceName, addresses] of Object.entries(networkInterfaces())) {
for (const address of addresses || []) {
if (address.family !== "IPv4" || address.internal || seen.has(address.address)) continue;
if (isTunnelInterface(interfaceName, address.address)) continue;
seen.add(address.address);
entries.push({
label: `Other devices (${interfaceName})`,
label: interfaceLabel(interfaceName, address.address),
url: `http://${address.address}:${port}`,
});
}
@@ -1,3 +1,6 @@
## System Architecture
```mermaid
flowchart TD
SOURCE[(Private inspection histories<br/>read-only source)]
SAMPLE[Private page-sampled histories<br/>up to 10,000 vehicles<br/>keyed vehicle tokens]
@@ -82,3 +85,4 @@ flowchart TD
class LOGISTIC,TREE model
class PUBLIC,CONTRACT,SITE,VIEWS,DEMO public
class GUARDRAILS warning
```
-192
View File
@@ -1,192 +0,0 @@
# Dashboard-only Bolt deployment
Last reviewed: 2026-07-21
> **Not part of the current delivery plan.** The presentation now uses the
> same-network live-demo procedure in [demo_script.md](demo_script.md). This
> checklist is retained only as a boundary reference if public hosting is
> reconsidered later.
## Non-negotiable publication boundary
Publish a **separate Bolt project or GitHub repository containing only the
contents of `dashboard/`**. In that project, `index.html` must be at the project
root.
Do not import the full Utah Vehicle Health source repository into Bolt. Do not
depend on Bolt supporting a `dashboard` working-root setting. The source
repository contains private-pipeline structure that a static host does not need,
even when local secrets and data are Git-ignored.
The deployment project must never contain `.env` files, credentials, private
data or marts, database files, model artifacts, SQL, pipeline scripts, repository
history from outside `dashboard/`, raw JSON, operational records, or row-level
predictions.
## 1. Verify the source dashboard locally
From the Utah Vehicle Health repository root:
```bash
npm --prefix dashboard test
node dashboard/server.mjs
```
Open `http://127.0.0.1:4173` and verify:
- the development-preview banner says the values come from the private
10,000-vehicle sample and are not population estimates;
- the routes are Overview, Sample cohorts, Model & benchmark, and Data & methods;
- the final model is calibrated logistic regression and the boosted tree is
benchmark-only;
- model cards show 2024 calibration-fit diagnostics, not the opened 2025
holdout comparison;
- no vehicle-level prediction service or input form exists;
- county displays communicate feed coverage rather than population rankings;
and
- a missing or invalid aggregate bundle produces an unavailable state, not
fallback estimates.
Stop the local server with `Ctrl-C` after verification.
## 2. Create a fresh dashboard-only artifact
Use a new directory and, preferably, a new deployment repository. Copy the
**contents** of `dashboard/`, not the directory's parent and not the main
repository's `.git` history.
The reviewed project tree is:
```text
index.html
styles.css
package.json
server.mjs
README.md
js/
app.js
charts.js
data.js
public/data/
age_risk_curve.json
cohort_scorecard.json
coverage_quality.json
data_manifest.json
filter_catalog.json
model_diagnostics.json
overview_period_county.json
sha256_manifest.json
tests/
contract.test.mjs
```
Do not add any other repository directory. In the fresh artifact root, run:
```bash
npm test
HOST=0.0.0.0 npm start
```
No dependency install or build step is required. The `PORT` environment
variable is honored automatically by `server.mjs` when a host supplies it.
Before creating the deployment repository, inspect its entire file tree. Apart
from `package.json` project metadata, the only JSON allowed is the reviewed
aggregate bundle under `public/data/`. Confirm that `data_manifest.json` still
declares:
```text
development_preview: true
population_estimate_allowed: false
locked_test_metrics_published: false
```
Do not hand-edit an aggregate file to make validation pass. Regenerate it
through the private local exporter and repeat the review.
## 3. Import only the dashboard repository into Bolt
The recommended handoff is a fresh GitHub repository containing the reviewed
tree above. It may be private if the connected Bolt account has access.
1. On the Bolt homepage, choose the GitHub import control.
2. Select the separate dashboard-only repository or use **Import from URL**.
3. Confirm `index.html` appears at the imported project root.
4. Inspect the Bolt file tree before previewing. If SQL, scripts, artifacts,
private data, `.env`, or source-repository files appear, stop and delete that
Bolt project; create a clean dashboard-only project instead.
5. Use `HOST=0.0.0.0 npm start` if Bolt requests a preview command. There is no
build command.
Bolt's current GitHub import workflow is documented in
[GitHub for version control](https://support.bolt.new/integrations/git).
If a separate GitHub repository is not used, create a fresh Bolt/StackBlitz
project and upload only the reviewed dashboard files so the same tree appears at
the project root. Never upload the full source repository and then try to hide
or ignore its private-pipeline directories.
## 4. Review in Bolt before publishing
Use the preview URL and repeat the local visual checks. Also verify directly:
- `/public/data/data_manifest.json` resolves;
- an unknown file path returns `404`;
- `/.env`, `/data/`, `/artifacts/`, `/models/`, `/sql/`, and `/scripts/` return
`404`;
- the browser makes requests only for the static HTML, CSS, JavaScript, and
approved files under `/public/data/`; and
- no analytics, database, authentication, or third-party data service was added
by an automated Bolt edit.
Preview privately first. If Bolt proposes code changes, review the full diff and
rerun `npm test`; do not accept changes that add a database, environment secret,
tracking script, model API, prediction form, or remote data source.
## 5. Publish with Bolt Hosting
Bolt Hosting is the intended host for this prototype:
1. Open the dashboard-only project.
2. Select **Publish** in the upper-right corner.
3. Choose private visibility for presentation review when available.
4. Select **Publish** and wait for the generated `bolt.host` URL.
5. Run the post-publication checks below before changing visibility to public.
Bolt documents this flow in
[Publish your project to a live website](https://support.bolt.new/cloud/hosting/publish).
Bolt's Netlify integration is optional and is not required for this dashboard;
see [Netlify integration](https://support.bolt.new/integrations/netlify) only if
hosting requirements explicitly change.
## 6. Post-publication checks
At the exact published URL:
1. Hard-refresh and confirm the development-preview banner remains visible.
2. Visit all four routes and verify labels, charts, and keyboard navigation.
3. Confirm the Model & benchmark view identifies logistic as final, tree as
benchmark, and its values as 2024 calibration-fit checks.
4. Confirm no 2025 holdout metric appears in the site or public JSON.
5. Confirm there is no vehicle-level lookup, input form, or row-level output.
6. Confirm coverage language does not imply statewide representation or county
ranking.
7. Request the denied paths from step 4 again and require `404`.
8. Save the published URL, release ID, publication date, and reviewer decision;
do not save private paths, row values, or credentials in the release note.
If any check fails, use Bolt's Publish menu to unpublish or keep the site private
until a corrected dashboard-only artifact passes local and hosted review.
## Updating a published prototype
Changes in a Bolt project are not automatically live. For each update:
1. regenerate/review aggregates locally;
2. run the dashboard contract test;
3. update the separate dashboard-only repository/project;
4. inspect Bolt's diff and preview;
5. use **Publish > Update**; and
6. repeat every post-publication check.
Never synchronize private pipeline directories into the deployment repository.
+1 -1
View File
@@ -112,7 +112,7 @@ possible, but must never appear in public outputs.
- Extreme repeat counts exist and require invalid/shared-identifier filtering.
Use a salted one-way internal token if a stable identifier is needed during
feature engineering. Never send VINs or plates to the browser or Bolt.
feature engineering. Never send VINs or plates to the browser.
## Operational and sensitive relations
-1
View File
@@ -222,4 +222,3 @@ ranking, production model, diagnosis, or individual decision tool.
- [Private-to-public architecture](architecture.mmd)
- [10-minute presentation outline](presentation_outline.md)
- [Dashboard demo script](demo_script.md)
- [Optional public-hosting notes](bolt_deployment.md)
+1 -1
View File
@@ -162,7 +162,7 @@ Use option 1 as the main story and option 6 as its data-engineering foundation:
3. Compare a transparent logistic baseline with a tree model.
4. Validate on a later time period and separately by county/source.
5. Export only aggregate scorecards, curves, and de-identified model outputs.
6. Build the public Bolt dashboard over those safe outputs.
6. Build the static dashboard over those safe outputs.
This gives the project a coherent end-to-end narrative across data engineering,
analytics, machine learning, responsible validation, and product design.
File diff suppressed because it is too large Load Diff