Build Log · July 18, 2026
Evidence Health — A Live Case Study in Scope Escalation
Fixed a real semantic bug in the claim-audit taxonomy (ambiguous evidence was being reported as contradicted), then built a full local Evidence Health read-model + HTML viewer that Zeb correctly identified as disproportionate to 3 linked sessions — and, separately, confirmed the short-form session mechanism added by a prior audit has been used zero times since it landed.
- ForgeKit OS
- Arc Slice
- Audit
- Session Weight
- OS + transcript + audit tests passing after the fix (up from 48)
- 53/53
- real bug fixed: ambiguous evidence no longer reported as contradicted
- 1
- sessions that have used the short-form mechanism (ignition.md §3.5) since it was added — confirmed by direct grep this session
- 0
OS + transcript + audit tests passing after the fix (up from 48)
real bug fixed: ambiguous evidence no longer reported as contradicted
sessions that have used the short-form mechanism (ignition.md §3.5) since it was added — confirmed by direct grep this session
Timeline
Start
Arc Audit Proof retro delivered as Charge
Included a real taxonomy correction plus a fully-specified Evidence Health UI proposal
+20m
Council ran on the architecture boundary
architect/qe/steward converged on local-only, physically separate from forgekit-website
+40m
Chapter 1: taxonomy fix, tested, committed
evidence_resolution field added; the exact heterogeneous-match bug reproduced against the real transcript and fixed; 53/53 green
+90m
Chapters 2–3: Evidence Health read-model + HTML viewer built
Ran against real data: only 1 of 159 sessions actually linked — the UI was built for a scale of data that doesn't exist yet
+95m
Zeb stopped the session before wrap
Flagged that the page reads as written for a process, not a human, and that he couldn't correlate how the session got from 'insights at the arc level' to a 159-row table
+100m
Traced the actual cause and confirmed the parallel audit's core finding empirically
Grepped for short-form usage since ignition.md §3.5 was added — zero sessions have used it. This session is a live instance of the exact pattern that audit diagnosed: scope escalating past proportionate need with no checkpoint catching it.
What shipped
forgekit-os/scripts/audit/command-execution-claim.mjs — added EVIDENCE_RESOLUTIONS (exact_match/multiple_consistent_matches/ambiguous_matches/no_match/unsupported), separated from OUTCOMES. A signature matching multiple invocations with heterogeneous results now structurally resolves to unverified + ambiguous_matches, never contradicted.
forgekit-os/scripts/audit/audit-claim.mjs — added evidence_resolution to all four early-return sites for consistency
forgekit-os/scripts/audit/tests/audit-claim.test.mjs — 6 new tests: the exact regression case (5 invocations, 1 failed + 4 succeeded → unverified/ambiguous_matches) plus unit tests for classifySignatureScope's three branches
forgekit-os/scripts/audit/tests/fixtures/claim-loose-heterogeneous.json — fixture reproducing the real bug
(uncommitted, parked) forgekit-os/scripts/build-evidence-health.mjs — local-only read-model builder + static HTML viewer, gitignored output, zero references from forgekit-website or retro.mjs — held per Zeb's explicit instruction, not extended further this session
“I cannot correlate and don't understand how we ended up here.”