Compare commits

..
16 Commits
Author SHA1 Message Date
kevinbell c6b36fa235 Merge pull request 'Add calibration slide and sharpen model-comparison evidence in deck' (#2) from deck-calibration-slide into main
Reviewed-on: #2
2026-08-27 23:00:44 +00:00
kevinbellandClaude Opus 5 180d58e646 Add calibration slide and sharpen model-comparison evidence in deck
Three changes to the presentation, all sourced from existing artifacts:

Slide 5 (model comparison): caption now states the evaluation set
(3,971 non-audit episodes from 3,734 vehicles, 487 non-pass), and a
callout reports top-10% precision of 31.2% against the 12.3% base
rate — about 2.5x lift. The two headline metrics alone gave the
audience no way to judge whether 0.261 PR-AUC is good.

Slide 6 (model decision): record that candidate selection happened on
the 2023 tune partition, where logistic led 0.282 to 0.274, and that
2024 fits Platt scaling only and so cannot select. The dashboard demo
displays 2024 numbers where the tree leads on both metrics; without
this line the deck has no answer to the obvious question.

New slide 7 (calibration): reliability plot of the 2025 holdout
deciles against the perfect-calibration diagonal, plotted from
artifacts/private/holdout/baseline_v1/calibration_bins.csv. Nothing in
the deck or dashboard previously showed calibration, though it is the
centerpiece claim of the final model.

Values verified against artifacts/private/holdout/*/metrics.json and
calibration_bins.csv. Deck remains self-contained; print/PDF backup
exports 11 pages.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 16:58:38 -06:00
kevinbell fbaf874f8d minor fixes 2026-08-27 16:47:09 -06:00
kevinbell 70bb5a4e07 Update README.md 2026-08-10 17:09:25 +00:00
kevinbell 27a483a86e Update README.md 2026-08-04 21:12:32 +00:00
kevinbell 98d0d24318 Update docs/architecture.md 2026-08-04 20:47:27 +00:00
kevinbell af987eb9e8 Update docs/architecture.md 2026-08-04 20:46:51 +00:00
kevinbell cbc80cf903 Update docs/architecture.mmd 2026-08-04 20:46:10 +00:00
kevinbell a3f64d44d9 added the macOS caffeinate command 2026-08-03 10:42:01 -06:00
kevinbell 70c37bc1b6 Update README.md 2026-08-03 16:37:34 +00:00
kevinbell f3b1843321 Update README.md 2026-07-31 18:39:38 +00:00
kevinbell 3ca57be97c website access minor edit 2026-07-29 11:43:18 -06:00
kevinbell d703f19ccc website access 2026-07-29 11:38:15 -06:00
kevinbell f2ab7f55f1 added presentation slides 2026-07-28 13:05:07 -06:00
kevinbell 7a9f4b6bc7 Update README.md 2026-07-28 17:23:31 +00:00
kevinbell fa2435388a clicking a tab no longer auto-scrolls 2026-07-27 14:44:17 -06:00
6 changed files with 1374 additions and 16 deletions
+11 -8
View File
@@ -1,3 +1,5 @@
<a name="readme-top"></a>
# Utah Vehicle Health # Utah Vehicle Health
Kevin Bell's summer 2026 development prototype for the first-attempt outcome of Kevin Bell's summer 2026 development prototype for the first-attempt outcome of
@@ -28,7 +30,7 @@ Start with the [final report](docs/final_report.md). Supporting deliverables:
- [Detailed development results](docs/development_results.md) - [Detailed development results](docs/development_results.md)
- [Model card](docs/model_card.md) - [Model card](docs/model_card.md)
- [Static dashboard specification](docs/dashboard_spec.md) - [Static dashboard specification](docs/dashboard_spec.md)
- [Private-to-public architecture](docs/architecture.mmd) - [Private-to-public architecture](docs/architecture.md)
- [10-minute presentation outline](docs/presentation_outline.md) - [10-minute presentation outline](docs/presentation_outline.md)
- [Dashboard demo script](docs/demo_script.md) - [Dashboard demo script](docs/demo_script.md)
@@ -150,6 +152,12 @@ repository root. Stop any older dashboard process with `Ctrl-C` first:
node dashboard/server.mjs --lan node dashboard/server.mjs --lan
``` ```
On macOS, to prevent the host from sleeping while the demo runs:
```bash
caffeinate -i node dashboard/server.mjs --lan
```
The server prints a **This computer** URL and one or more interface-labeled The server prints a **This computer** URL and one or more interface-labeled
**Other devices** URLs. Open the URL for the shared Wi-Fi or Ethernet interface **Other devices** URLs. Open the URL for the shared Wi-Fi or Ethernet interface
on the second computer; do not use `0.0.0.0` as the browser address. Confirm the on the second computer; do not use `0.0.0.0` as the browser address. Confirm the
@@ -162,13 +170,6 @@ page, so the dashboard includes a dependency-free checksum implementation for
that case. These checks validate bundle consistency; plain HTTP does not that case. These checks validate bundle consistency; plain HTTP does not
authenticate the network transport, so use a trusted demo network. authenticate the network transport, so use a trusted demo network.
## Public hosting is not required
The presentation plan is the same-network live demo above. Nothing needs to be
uploaded or published, and the server exposes only the dashboard's explicit
static-file allowlist. If public hosting is ever reconsidered, publish only a
separately reviewed copy of the `dashboard/` directory—never the private source
repository.
## Author ## Author
@@ -177,3 +178,5 @@ Kevin Bell
<kevinBell@Linux.com> <kevinBell@Linux.com>
<https://kevinbell.dev/> <https://kevinbell.dev/>
<p align="left"><a href="#readme-top">back to top</a></p>
+4
View File
@@ -41,6 +41,10 @@ address, not the address to put in a browser. Confirm the in-app status reads
**Validated sample aggregates**, then stop the server with `Ctrl-C` after the **Validated sample aggregates**, then stop the server with `Ctrl-C` after the
demo. demo.
If the host is on a Tailscale tailnet, the server also prints a **Tailscale
(tunnel)** URL (a `100.x.y.z` address). Open that URL on any other device signed
into the same tailnet to reach the dashboard remotely, without sharing a LAN.
All data-contract and checksum validation remains enabled over LAN HTTP. When a All data-contract and checksum validation remains enabled over LAN HTTP. When a
remote browser does not expose the SubtleCrypto digest API to the plain-HTTP remote browser does not expose the SubtleCrypto digest API to the plain-HTTP
page, the dashboard uses its dependency-free SHA-256 implementation and still page, the dashboard uses its dependency-free SHA-256 implementation and still
+1 -1
View File
@@ -86,8 +86,8 @@ function showRoute({ announce = true, focus = false } = {}) {
if (heading) { if (heading) {
heading.setAttribute("tabindex", "-1"); heading.setAttribute("tabindex", "-1");
heading.focus({ preventScroll: true }); heading.focus({ preventScroll: true });
heading.scrollIntoView({ behavior: "smooth", block: "start" });
} }
window.scrollTo({ top: 0, left: 0, behavior: "instant" });
} }
} }
+7 -7
View File
@@ -52,12 +52,13 @@ function safeFile(requestUrl) {
} }
} }
// Tailscale and similar VPN tunnels show up as extra "LAN" interfaces but are not // A tailnet advertises the host over a 100.64.0.0/10 address, so label that URL
// useful for an in-room demo, so keep them out of the printed URL list. // clearly for remote (off-LAN) demos over the tunnel.
function isTunnelInterface(interfaceName, address) { function interfaceLabel(interfaceName, address) {
if (/^(utun|tailscale|tun|ppp|ipsec)/i.test(interfaceName)) return true;
const [first, second] = address.split(".").map(Number); const [first, second] = address.split(".").map(Number);
return first === 100 && second >= 64 && second <= 127; // 100.64.0.0/10 (CGNAT/tailnet) const isTailnet =
/tailscale/i.test(interfaceName) || (first === 100 && second >= 64 && second <= 127);
return isTailnet ? "Remote (tunnel)" : `Other devices (${interfaceName})`;
} }
function displayUrls(host, port) { function displayUrls(host, port) {
@@ -72,10 +73,9 @@ function displayUrls(host, port) {
for (const [interfaceName, addresses] of Object.entries(networkInterfaces())) { for (const [interfaceName, addresses] of Object.entries(networkInterfaces())) {
for (const address of addresses || []) { for (const address of addresses || []) {
if (address.family !== "IPv4" || address.internal || seen.has(address.address)) continue; if (address.family !== "IPv4" || address.internal || seen.has(address.address)) continue;
if (isTunnelInterface(interfaceName, address.address)) continue;
seen.add(address.address); seen.add(address.address);
entries.push({ entries.push({
label: `Other devices (${interfaceName})`, label: interfaceLabel(interfaceName, address.address),
url: `http://${address.address}:${port}`, url: `http://${address.address}:${port}`,
}); });
} }
@@ -1,3 +1,6 @@
## System Architecture
```mermaid
flowchart TD flowchart TD
SOURCE[(Private inspection histories<br/>read-only source)] SOURCE[(Private inspection histories<br/>read-only source)]
SAMPLE[Private page-sampled histories<br/>up to 10,000 vehicles<br/>keyed vehicle tokens] SAMPLE[Private page-sampled histories<br/>up to 10,000 vehicles<br/>keyed vehicle tokens]
@@ -82,3 +85,4 @@ flowchart TD
class LOGISTIC,TREE model class LOGISTIC,TREE model
class PUBLIC,CONTRACT,SITE,VIEWS,DEMO public class PUBLIC,CONTRACT,SITE,VIEWS,DEMO public
class GUARDRAILS warning class GUARDRAILS warning
```
File diff suppressed because it is too large Load Diff