Build Log

Evidence Health — A Live Case Study in Scope Escalation

July 18, 2026

Fixed a real semantic bug in the claim-audit taxonomy (ambiguous evidence was being reported as contradicted), then built a full local Evidence Health read-model + HTML viewer that Zeb correctly identified as disproportionate to 3 linked sessions — and, separately, confirmed the short-form session mechanism added by a prior audit has been used zero times since it landed.

ForgeKit OSArc SliceAuditSession Weight

53/53

OS + transcript + audit tests passing after the fix (up from 48)

real bug fixed: ambiguous evidence no longer reported as contradicted

sessions that have used the short-form mechanism (ignition.md §3.5) since it was added — confirmed by direct grep this session

Timeline

Start

Arc Audit Proof retro delivered as Charge

Included a real taxonomy correction plus a fully-specified Evidence Health UI proposal

+20m

Council ran on the architecture boundary

architect/qe/steward converged on local-only, physically separate from forgekit-website

+40m

Chapter 1: taxonomy fix, tested, committed

evidence_resolution field added; the exact heterogeneous-match bug reproduced against the real transcript and fixed; 53/53 green

+90m

Chapters 2–3: Evidence Health read-model + HTML viewer built

Ran against real data: only 1 of 159 sessions actually linked — the UI was built for a scale of data that doesn't exist yet

+95m

Zeb stopped the session before wrap

Flagged that the page reads as written for a process, not a human, and that he couldn't correlate how the session got from 'insights at the arc level' to a 159-row table

+100m

Traced the actual cause and confirmed the parallel audit's core finding empirically

Grepped for short-form usage since ignition.md §3.5 was added — zero sessions have used it. This session is a live instance of the exact pattern that audit diagnosed: scope escalating past proportionate need with no checkpoint catching it.

What shipped

forgekit-os/scripts/audit/command-execution-claim.mjs — added EVIDENCE_RESOLUTIONS (exact_match/multiple_consistent_matches/ambiguous_matches/no_match/unsupported), separated from OUTCOMES. A signature matching multiple invocations with heterogeneous results now structurally resolves to unverified + ambiguous_matches, never contradicted.

forgekit-os/scripts/audit/audit-claim.mjs — added evidence_resolution to all four early-return sites for consistency

forgekit-os/scripts/audit/tests/audit-claim.test.mjs — 6 new tests: the exact regression case (5 invocations, 1 failed + 4 succeeded → unverified/ambiguous_matches) plus unit tests for classifySignatureScope's three branches

forgekit-os/scripts/audit/tests/fixtures/claim-loose-heterogeneous.json — fixture reproducing the real bug

(uncommitted, parked) forgekit-os/scripts/build-evidence-health.mjs — local-only read-model builder + static HTML viewer, gitignored output, zero references from forgekit-website or retro.mjs — held per Zeb's explicit instruction, not extended further this session

“I cannot correlate and don't understand how we ended up here.”
— Zeb, on the Evidence Health UI — the question that ended the session's build phase and reframed it as a case study instead