Build Log

The Armory — Building Real Functional Test Coverage After a Production Outage

August 19, 2026

A production outage from an invalid Server Actions export led to a full risk survey of all 95 MedinaCHF server actions, a real functional Playwright test suite proven to catch the bug that started it, live GitHub Actions CI, and four real bugs (two security gaps, one silent-failure bug, one production 500) found and fixed along the way.

MedinaCHFTestingCI/CDSecurityProduction Incident

real functional Playwright specs added, covering ~35 server actions

server actions surveyed and risk-ranked across 25 lib/actions/*.ts files

real bugs found and fixed (2 security/IDOR gaps, 1 silent-failure email bug, 1 production 500)

false-negative armory specs — every spec proven to actually fail on the bug it claims to catch

Timeline

Start

Took over uncommitted work from a session that lost its connection — real, in-progress MedinaCHF member-invitation and admin-nav features

Worked in a git worktree per concurrency.md, verified with tsc + a live DB check, merged into master, deployed

Report 1

Zeb reported: 'when I look at the event from the coordinator view it shows no planner is assigned'

Traced to createEvent/updateEvent deriving isCoordinator from Clerk role alone — a coordinator using the member-facing Plan page still hit the coordinator branch. Fixed with a formAudience form field the server trusts over role.

Report 2

Zeb reported: 'adding a member is broken' in production

A 'use server' file (member-management.ts) exported a plain number — crashed the whole module at runtime. Neither tsc nor a real next build caught it; only invoking the action did. Fixed and deployed.

Charge

Zeb: 'I want a suite of tests for all functionality. it will run every build.'

Scoped via AskUserQuestion: CI on push/PR (not just local predeploy), coverage prioritized by risk, not exhaustiveness

Survey

Dispatched an Explore agent to map and risk-rank all 95 exported server actions across 25 files

Produced a per-file risk breakdown (AUTH_ACCESS/SCHEMA_MUTATION/EXTERNAL_SERVICE/READ_ONLY/COSMETIC) used to sequence Tier 1/2/3 coverage

Tier 1

Built and proved 7 specs against the highest-risk untested functions

deleteMember (19-table invariant), updateEvent (untested twin of the formAudience bug), deleteEvent, transitionApplication (full state machine), publishOpportunityAsEvent — found and fixed a real sendEmail bug along the way

CI stood up

Built .github/workflows/medinachf-tests.yml — typecheck, build, then the full armory on every push/PR

Required Zeb to add 4 GitHub Secrets; first real run failed on a stale secret, second attempt confirmed green end-to-end

Tier 2/3

Built 20 more specs across email changes, RSVP, attendance, community, polls, payments, announcements

Found and fixed two more real auth gaps (updateClaimProgress, addPollOption — both had zero auth checks despite siblings having them)

CI hardening

Two real CI-only failures found and fixed: a timezone bug in a test (wrong-timezone 'past event' computation) and a genuine apt-mirror stall in the Playwright browser install step

Dropped --with-deps entirely after a 3rd occurrence of the same stall; added a 5-minute timeout as a backstop

27/27 specs passing locally and in CI; a real CI/local Clerk-instance collision observed and documented (self-healing, not fixed structurally)

What shipped

Recovered and shipped a lost session's member-invitation/resend workflow + admin mobile nav + Plan page rework (products/medinaCHF)

Fixed a real formAudience bug in createEvent/updateEvent — a coordinator using the member-facing Plan page got no planner assigned

Fixed a real production outage — an invalid non-async export crashing member-management.ts's whole module at runtime

Rebuilt the dev Neon database from a fresh production snapshot (dev-2026-08-18 branch) after the old one was reported broken

products/medinaCHF/tests/e2e/armory/ — 27 real functional Playwright specs (not screenshot-only) covering ~35 of 95 surveyed server actions, prioritized by risk

products/medinaCHF/scripts/gate-build.mjs + npm run gate:build — real production-build CI check

products/medinaCHF/scripts/cleanup-armory-data.mjs + npm run test:e2e:cleanup — resets accumulated armory test data

.github/workflows/medinachf-tests.yml — typecheck + build + full armory on every push/PR touching products/medinaCHF

Fixed 4 real bugs found while building coverage: lib/email.ts's sendEmail letting a missing API key crash the caller instead of degrading gracefully; updateClaimProgress and addPollOption both had zero auth checks (IDOR-shaped gaps); the original production-outage export bug

“keep going”
— Zeb, after being shown the full Tier 1 armory results and asked whether to continue to Tier 2/3