A passing test suite answers an important question: does the implementation behave the way the tests expect? It does not necessarily answer the harder question: does the product still honor the intent that caused us to build it?
The most valuable failure on our dashboard
While reconstructing the verification record for the Ross Beurmann Consulting platform, we found two failed acceptance criteria in an older specification. At first glance, that sounds like ordinary regression debt. It was not.
SPEC-006 described the pre-launch contact experience. It deliberately offered a contact address and collected no visitor data. At the time, those criteria were correct. The implementation met them.
Later, SPEC-009 introduced a real lead-capture workflow. It intentionally collected and persisted contact submissions. The new feature worked as designed—but its instantiation made two formerly valid SPEC-006 criteria false.
We had several ways to make the dashboard green. We could delete the old criteria. We could rewrite SPEC-006 as though it had always anticipated lead capture. We could ignore the failure because the new feature was intentional.
Every one of those choices would destroy useful information.
The failure told us exactly when the product’s intent changed, which later decision changed it, and whether the discrepancy was a defect or an approved replacement. That is not test noise. That is product intelligence.
SDD needs an evidence contract
Spec-Driven Development gives AI-assisted work a much stronger foundation than conversational prompting. The specification creates reviewable intent. Acceptance criteria constrain interpretation. Implementation can be traced back to a reason.
But a specification alone does not guarantee that its acceptance criteria are provable, that yesterday’s evidence still applies today, or that a new specification has not invalidated older intent. If verification arrives after implementation, teams discover those problems at the most expensive point in the flow.
We call our response Verification-First Spec-Driven Delivery, or VF-SDD. We are not claiming to have invented verification-first engineering or Spec-Driven Development. VF-SDD is RBC’s specific operating method for connecting specification intent, evidence design, dependency change, release control, and visible rework.
Evidence method
How the criterion will be proven: automated test, browser, API, database, security review, or durable human evidence.
Evidence environment
Where that proof is valid: repository, CI, preview, production, database, or a controlled manual review.
Release severity
Whether missing or contradictory evidence blocks deployment or enters an advisory rework queue.
Failure response
Repair the implementation, amend the intent, supersede it, retire it, or collect the missing evidence.
What changes when verification comes first
Under VF-SDD, every acceptance criterion becomes an evidence row before implementation begins. It must say how proof will be produced, where that proof is valid, whether its failure blocks release, and what decision should follow if it does not pass.
The specification itself is revision-pinned to its verification manifest. Declared specification dependencies are pinned too. If either changes, the old evidence becomes stale. The system does not assume that a passing result from an earlier version still proves the current intent.
That creates two deliberately separate controls:
- The pre-deployment gate fails closed for stale evidence, malformed contracts, invalid dependency pins, and non-passed criteria designated as release-blocking.
- The rework queue retains partial, missing, blocked, and failed advisory evidence without pretending that every evidence gap is a production defect.
This distinction matters. A release gate that blocks on everything becomes bureaucracy and is eventually bypassed. A dashboard that blocks on nothing becomes decoration. VF-SDD makes the risk decision explicit at the criterion level.
Five honest rework decisions
A failed criterion should not automatically generate “fix the code.” The implementation may be wrong, the specification may be wrong, the evidence may be incomplete, or later intent may have legitimately replaced the earlier requirement.
- Repair: the implementation drifted from valid intent.
- Amend the spec: the written intent is incorrect or incomplete.
- Supersede: approved successor intent replaced the earlier criterion.
- Retire: the capability was intentionally removed without a successor.
- Collect evidence: the behavior may be correct, but the proof is incomplete.
SPEC-006 required supersession, not a rollback of SPEC-009. Preserving the failed observation alongside the supersession decision gives future maintainers both the historical truth and the current product truth.
Why this matters more with AI
AI can produce implementation, tests, and documentation at extraordinary speed. It can also reproduce the same misunderstanding across all three artifacts. A coherent stack of spec, code, and tests may still be coherently wrong.
Verification-first design forces the team to decide what independent evidence would be convincing before the implementation starts shaping the answer. It also reduces a subtler risk: delivery moving so quickly that evidence becomes permanently attached to an earlier version of intent.
The goal is not more paperwork. The goal is a repository that can answer, at any moment:
- What did we intend?
- What proves it?
- Is that proof still current?
- What changed the intent?
- What must happen before release?
- What remains as deliberate rework?
The spec is the scoreboard. Evidence keeps score.
Our public delivery dashboard currently shows both a passing pre-deployment gate and an advisory rework queue. That is not a contradiction. It is the point.
The gate says the current release met the evidence obligations designated as blocking. The queue says we still owe evidence, disposition, or controlled follow-up elsewhere. The historical-intent event says a later specification intentionally changed what an earlier specification required.
That is a far more useful picture than “all tests passed.” It tells us whether the product, the evidence, and the decisions that shaped both are still aligned.
RBC is developing VF-SDD in public through the same repository controls the method prescribes.
Living field notes
Addenda from the verification queue
VF-SDD is being refined against the same delivery system it governs. These dated notes preserve what the queue teaches us without quietly rewriting the original argument.
Addendum 001 · August 29, 2026 · SPEC-001
Evidence has to be fit for the claim
SPEC-001’s reconstructed manifest cited a header unit test as evidence for whole-site responsive reflow, visible keyboard focus, and reduced-motion behavior. The test existed and passed. It was still incapable of proving those claims.
The disposition split the work honestly: repair and automate route-close behavior, then collect browser evidence at the viewport and preference boundaries where reflow, focus, and motion actually occur. VF-SDD therefore needs more than an evidence reference. It needs an evidence method capable of falsifying the criterion.
Addendum 002 · August 29, 2026 · SPEC-004
An evidence gap became an observed defect
SPEC-004 already carried an advisory item for incomplete mobile-browser evidence. A real-device review then showed the case-study metadata rail colliding with headlines in portrait orientation while the intended four-column composition remained strong in landscape.
No new criterion was required and the queue did not need to grow. The missing evidence had done its job: it marked uncertainty until observation converted that uncertainty into a specific repair. The portrait layout now stacks metadata above the headline; the landscape and desktop composition remains unchanged.
Addendum 003 · August 30, 2026 · SPEC-002
Evidence should become infrastructure
SPEC-002’s mobile-reflow criterion had been linked to a component test. That test could confirm content and semantics, but it could not render CSS, measure the document, or observe a browser console. The evidence row therefore remained partial even while the homepage looked correct.
The resolution was a reusable production-browser contract in CI. It loads the built site at 390 and 320 CSS pixels, asserts the intended responsive columns, fails on horizontal overflow, and captures browser errors. Closing one evidence gap created a verification capability that later specifications can reuse instead of collecting bespoke proof after every release.
Addendum 004 · August 30, 2026 · SPEC-003
Good evidence compounds
SPEC-003 carried the same browser-evidence gap as the homepage: its component tests proved service content, but not responsive layout, overflow, or runtime health. This time the answer was not another bespoke verification effort.
The production-browser capability created for SPEC-002 was extracted into shared assertions and extended to the Services route. The marginal cost of verifying another specification fell while the evidence became stronger and more consistent. Verification infrastructure behaves like product infrastructure: designed well, each use makes the next release cheaper to trust.
Addendum 005 · August 30, 2026 · SPEC-004
A repair is not closed by the repair
The portrait metadata collision found during SPEC-004 review was repaired immediately, but its acceptance criterion stayed partial. A code change proved that we responded; it did not prove that the repaired layout worked across the Work index and every case-study route.
Closure required two independent layers: case-specific assertions for the approved evidence and caveats, then a production-browser regression across all six routes at desktop, 390 pixels, and 320 pixels. The first run also rejected an incorrect desktop expectation in the verifier itself: the product correctly used a grid while the test expected a block. VF-SDD treats implementation as the candidate answer, and verification code as another claim that must survive evidence.
Addendum 006 · August 30, 2026 · SPEC-005
Absence needs evidence too
SPEC-005 promised that assistant conversations and leads were not persisted. A privacy sentence in the interface described the intent, but it did not prove the absence of a storage path.
The evidence contract now combines a forbidden-dependency and browser-storage boundary test with an end-to-end browser check: submit a locally grounded question, render the approved response, reload, and confirm the transcript is gone. Negative requirements deserve positive verification design.
Addendum 007 · August 30, 2026 · SPEC-005 → SPEC-011
A stronger contract should disturb its dependents
Strengthening SPEC-005 changed its specification revision. The pre-deployment gate then stopped the release because SPEC-011, the approved-claims contract used by the assistant, still pinned the previous revision.
The assistant tests had passed, but that was not enough. We reviewed the dependency deliberately, confirmed that the privacy and interaction changes did not alter the approved-claims boundary, and pinned the new revision. Dependency friction is useful when it forces intent to be reconsidered instead of inherited silently.
Addendum 008 · August 30, 2026 · SPEC-006 → SPEC-009
Closed does not always mean passed
SPEC-006 required a pre-launch contact state with no collection or persistence. SPEC-009 deliberately replaced that behavior with privacy-conscious lead capture. Making the older criteria pass now would require breaking the current product—or rewriting the record.
VF-SDD keeps those two criteria failed and records their approved supersession separately. The surviving responsive-accessibility criterion is verified against the successor contact experience. A failed observation can represent a closed decision when the disposition is explicit, while the historical signal remains available for audit and learning.
Addendum 009 · August 30, 2026 · SPEC-007 → SPEC-018
Publication changes the evidence boundary
SPEC-007 established Thinking as an honest pre-publication home: themes could be visible, but drafts could not masquerade as finished work. Publishing the VF-SDD field note through SPEC-018 made that lifecycle description stale even though the original honesty criterion still held.
The reconciled contract now verifies two different claims: the themes remain clearly framed as areas of inquiry, and the article link points to a real governed publication. Evidence must evolve when a placeholder becomes a product surface; otherwise yesterday's truthful test can become tomorrow's incomplete one.
Addendum 010 · August 30, 2026 · SPEC-008
Production readiness is a graph property
A handful of passing page tests could not prove the launch surface. Sitemap generation, metadata uniqueness, canonical URLs, indexing policy, policy-page access, social assets, unknown-route recovery, narrow reflow, and runtime health are connected claims.
SPEC-008 now derives its browser crawl from the generated sitemap and verifies every declared indexable route at 320 pixels. The evidence grows with the product surface automatically. Readiness is not a badge attached to one page; it is a property of the route graph and the contracts connecting it.
Addendum 011 · August 30, 2026 · SPEC-009
Privacy claims need adversarial proof
SPEC-009 said browser roles could not read or write lead records. The server-route tests proved that a secret-bearing server request could persist a lead, but success through the privileged path said nothing about denial through the public paths.
The evidence contract now asks the connected database directly: RLS must be enabled, both anon and authenticated must hold zero read or write grants, and no policy may reopen access. We then submit one uniquely tagged synthetic lead through production and delete that exact record. Privacy is proved by testing both the path that must work and the paths that must not.
Addendum 012 · August 30, 2026 · SPEC-010
One negative promise can hide two evidence obligations
“Conversations are not saved” sounded like one privacy requirement. The existing boundary test proved there was no Supabase, browser-storage, insert, or upsert path. It did not prove that prompt and response bodies were absent from operational logs.
SPEC-010 now splits the promise into independent falsifiable checks: one scans for persistence paths; another constrains log calls to operational metadata and executes requests containing unique markers to prove those markers never reach captured logs. Negative requirements should be decomposed by every place the forbidden data could escape.
Addendum 013 · August 30, 2026 · SPEC-011
Source truth and placement truth are different claims
The approved-claims tests proved that the controlled register contained 29 unique claims and excluded retired wording. They did not prove that About, Work, and Services rendered the intended curated subsets.
The new placement contract resolves every page's claim IDs through the controlled dataset, then asserts exact wording and exact counts inside each named proof region. A trustworthy source can still be routed incorrectly. Content governance therefore needs evidence for both what is approved and where the approved material actually appears.
Addendum 014 · August 30, 2026 · SPEC-018 + SPEC-019
Observation should change the contract before the code
A production review exposed two interaction defects that repository checks had not represented: desktop navigation used a different interaction model from mobile, and the article's application banner was visually absent at rest and placed after the field addenda.
We recorded each observation as a failed advisory evidence row before beginning repair. The queue grew by two on purpose. This preserves the moment the product contradicted newly clarified intent, makes the repair target testable, and lets later before-and-after evidence close the findings without erasing how they were discovered.
Addendum 015 · August 30, 2026 · SPEC-012
One implementation can hide five evidence repairs
SPEC-012 displayed five open rows, yet production inspection found the security headers, cleanup job, privacy language, advisor boundary, and complete CI flow largely operating as intended. The manifest had copied one database migration across claims it could not prove.
Closure required only one material test improvement: exercising the approved red tokens against every intended surface. The rest was evidence replacement and lifecycle reconciliation, including retiring a pre-launch protection condition after the public cutover was authorized. Rework count is not defect count; verification has to diagnose the difference.
Addendum 016 · August 30, 2026 · SPEC-013
Private systems need synthetic public proof
SPEC-013 could not honestly prove its authorized CRM flow with a public browser suite, and production lead data could not become test material. The result was a familiar trap: strong manual confidence paired with weak repeatable evidence.
We introduced a local Supabase contract double containing approved and unapproved identities plus 26 synthetic leads, then rebuilt the database baseline in an ephemeral PostgreSQL service. CI can now prove denial, pagination, date filtering, detail rendering, responsive behavior, RLS, grants, and indexes without exposing a single production lead. Verification quality does not require sacrificing privacy; it requires designing a safe evidence environment.
Addendum 017 · August 30, 2026 · SPEC-014
Implementation completeness and evidence completeness are different
SPEC-014 already had working status, notes, follow-up, activity, and concurrency code. Its manifest still treated manual production observations and source presence as proof for authorization, immutability, failure behavior, and public-data isolation. The feature looked nearly complete because the implementation was nearly complete; the evidence was not.
Closure required one real behavior repair—terminal leads no longer inflate the overdue queue—and a broader evidence reconstruction. An ephemeral database now proves exact activity events, authorized actors, stale-write rejection, immutable submissions, append-only history, and rollback to zero synthetic records. The browser proves success, pending, conflict, failure, empty, responsive, and keyboard states against synthetic leads. VF-SDD has to measure both what the product does and how convincingly we can prove it.
Addendum 018 · August 31, 2026 · SPEC-015
Provider acceptance and delivery are different truths
Microsoft Graph returns acceptance for processing, not proof that a message reached an inbox. Treating that response as “delivered” would make the CRM more confident than its evidence.
SPEC-015 therefore stores the provider result as accepted, preserves Microsoft request correlation, and leaves delivery unset. Closure combined automated failure-path and privacy contracts with Exchange mailbox scoping, one controlled send, one CRM record, Sent Items presence, and independent recipient observation. Evidence can be layered without collapsing distinct claims into one status.
Addendum 019 · August 31, 2026 · SPEC-016
Observability must preserve its last trustworthy answer
A delivery dashboard that overwrites good data with a failed collection does not expose uncertainty; it publishes corruption. SPEC-016 now writes its public artifact only after the complete verification contract succeeds, so collector failure remains visible to maintainers while readers retain the last known good result.
The same closure made dependency edges and critical-path nodes carry repository source references instead of appearing as unexplained diagram facts. Public delivery truth needs provenance for relationships as much as it needs honest labels for metrics.
Addendum 020 · August 31, 2026 · SPEC-018
The article has to survive its own method
A production review found this article's application banner after the addenda and visually absent at rest because a secondary link inherited dark text on a dark background. The link existed and remained clickable; neither fact proved that readers could see it.
The observation entered the verification manifest before repair. The banner now precedes the append-only field notes, uses an explicit high-contrast treatment, and is exercised at desktop and 320 CSS pixels for visibility, focus, ordering, overflow, and runtime errors. Thought leadership about verification should be governed by the same evidence contract it advocates.
Addendum 021 · August 31, 2026 · SPEC-019
Parity means one interaction contract
The header already contained one accessible Menu control and all nine shared destinations, but desktop CSS hid the control and exposed only the five primary links inline. Component tests saw the complete DOM and passed while production users received two different navigation models.
SPEC-019 moved the Menu interaction into the base layout and made viewport differences presentational only: a compact desktop panel and a mobile list open from the same trigger, expose the same destinations, and close through the same behavior. Cross-viewport browser evidence now tests what users can actually see and operate, not merely what React rendered.
Addendum 022 · August 31, 2026 · SPEC-020
A valid token is not the whole authorization decision
The CRM already asked Supabase to validate every session token, but its surrounding account lifecycle was thinner than the route check suggested. A hard-coded fallback administrator survived missing configuration, sign-out removed only the local cookie, recovery was absent, and provider hardening remained disabled.
SPEC-020 separates identity, authorization, session state, and account lifecycle into independently falsifiable claims. Provider validation proves the token; a server-owned allowlist proves current authority; global revocation and short sessions bound access; recovery and offboarding prove how authority changes. Authentication is not one successful login—it is a continuously governed decision.
Addendum 023 · August 31, 2026 · SPEC-021
Unavailable is a measurement state, not a zero
Operational telemetry tempted us into an easy lie: when no production failure or recovery event existed, the dashboard could have displayed a change failure rate of 0% and a recovery time of 0 seconds. Both numbers would have looked healthy. Neither would have been observed.
SPEC-021 makes absence explicit. Deployment frequency and lead time can be derived from observed production deployments, while change failure rate and recovery time remain unavailable until their required failure and paired recovery evidence exists. A useful dashboard distinguishes “nothing happened” from “we have not observed enough to calculate this.”
Addendum 024 · August 31, 2026 · SPEC-021
A model is not a bill
Token counts can support a modeled cost, but they do not prove what a provider actually charged. Publishing that calculation as “token spend” would collapse two different evidence classes into one convenient number.
SPEC-021 therefore versions the pricing configuration, labels the result modeled, exposes the observation window and token totals, and refuses to manufacture $0.00 when no usage events exist. Provider-billed cost can only become observed when billing evidence supports it. Financial telemetry deserves the same evidence discipline as product telemetry.
Addendum 025 · August 31, 2026 · SPEC-021 → SPEC-022
A stale dependency pin is useful friction
Closing SPEC-021 changed its specification revision. The next verification run failed before build or browser work because SPEC-022 still pinned the previous revision. The application had not broken; the evidence graph had detected a downstream contract that had not been reconsidered.
We reviewed the relationship, refreshed the pin, and reran the complete gate. This is exactly the kind of failure teams are tempted to dismiss as CI bookkeeping. VF-SDD treats it as intent propagation: when upstream truth changes, downstream confidence should be disturbed until someone explicitly accepts the new dependency.
Addendum 026 · August 31, 2026 · SPEC-021
Collateral regressions are still regressions
Moving unavailable DORA measures into the operational telemetry row looked like a presentation-only cleanup. Unit tests, manifest verification, database reconstruction, and the production build all passed. The browser gate still failed because removing an empty evidence section also removed the existing “Read the case study” action.
The repair restored the action without undoing the telemetry improvement. Verification should not be scoped only to the thing we meant to change; it must also protect contracts that share the changed surface. A passing implementation story can still violate a neighboring user interaction.
Addendum 027 · August 31, 2026 · SPEC-023
Optimize iteration, not the standard of proof
By SPEC-023 the repository had 23 manifests, 126 unit tests, database reconstruction, a production build, and 41 browser flows. Running that entire product regression after every small draft commit was safe but increasingly wasteful—and made fast evidence feel like bureaucracy.
The next control makes the verification graph executable. Draft work can select tests from changed files, versioned impact mappings, and recursive specification dependencies; shared or unmapped changes expand rather than shrink the scope. Ready-for-review work and verification closure still require the complete repository gate. Dependency-aware verification changes how quickly we iterate, not what “verified” means.