“How can you be confident in code you did not read or write in full?”
Confidence needs a trail of evidence: a clear requirement, an answer specified separately from the implementation, and a check that can expose a mistake. Reading code is part of that work. This small example shows what the check adds.
EXPERIMENT 0 / INVENTED HARBOR DISPATCH CASE
Read the code. Then check the answer.
The rule: include paid, unshipped orders at least 48 hours old. Exclude cancellations and repeated IDs.
Inspect the code for mismatches with that rule. Then run the check.
IMPLEMENTATION / SQL
WITH eligible_orders AS (
SELECT
order_id
FROM dispatch_orders
WHERE paid = 1
AND shipped = 0
AND age_hours > 48
AND canceled = 0
)
SELECT order_id
FROM eligible_orders
ORDER BY order_id;
EXPECTED / SPECIFIED SEPARATELYA102 · A105
A102 is exactly 48 hours old. A105 is 60 hours old. Each belongs once.
Not run yet
The expected IDs are fixed before this implementation is checked.
The check compares order IDs and multiplicity, not just the total. Reading and testing supply different evidence.
Inspect the six invented input rows
Order
Age
State
Purpose
A101
47 hours
Paid, unshipped
One hour too early
A102
48 hours
Paid, unshipped
Exactly at the boundary
A103
72 hours
Canceled
Canceled, despite its age
A104
60 hours
Shipped
Already shipped
A105
60 hours
Paid, unshipped
Eligible order
A105
60 hours
Paid, unshipped
Repeated copy of the same order
Harbor Dispatchinvented teaching casesdisplayed code executes locally; expected IDs are separate from the implementation
01The V
1. Pair every promise with proof
The missing order is one broken link in the V: “at least 48 hours” did not survive implementation. Read down the left to define the work, then up the right to check it. Each expected result traces to a requirement, not to the code being tested.
THE V MODEL / ADAPTED FOR ANALYTICS
Specify the promise. Plan its proof.
Select a matching pair
DEFINE ↓ before building
CHECK ↑ after building
Each check traces back to its promise
4 / IMPLEMENTATIONBuild the queueAI agents work to the specification
Decision rule → known answers
A102 and A105 should appear once each.
Solid arrows show the work sequence. Dashed links pair a requirement with its check. Define the expected evidence before coding; perform and review the checks afterward.
I learned this at Philips, on Class III medical devices. The quality owner who signed off the software had not written it. They read the requirements and the checks tied to them, then took responsibility for acceptance. That is the V: accountability supported by evidence linked to requirements.
AI changed who sits in the V. At Philips, separation came from different people: an analyst wrote the requirements, a developer built, and a tester checked. I built my analytics portfolio using AI tools. When one person and a set of agents do all of it, I have to construct that separation deliberately. The operating model starts with expected answers before implementation, a second path that receives rules and raw cases without the first implementation, and an accountable owner who accepts the intended use.
AI also makes another implementation easier to produce. I can give a second agent the rules document and raw cases without the first implementation or output. The work that remains is mine: choosing cases that could fail, naming shared assumptions, reviewing the second path and reading any disagreement.
Humans still read code. The review split is explicit: I read SQL, configuration and tests, where the business rule is expressed and checked. An engineer reviews the Python internals, scaled to what a wrong number would cost. The accountable owner reads the requirement in their own words and the evidence against it, then decides whether to accept the result for that use.
Proposed operating model: the review split and separate owner acceptance describe how the work should be organized, not completed portfolio sign-offs.
Same V, less ceremony
Level
Regulated software (the V with full ceremony)
AI-assisted analytics (the same V)
Intended use
Approved use statement
Owner question, scope and acceptance
Rule
Controlled requirements
Versioned rule and expected cases
Specification
Design record and traceability matrix
Data contract and requirement-to-test links
Code
Reviewed release baseline
Versioned implementation and assigned review
Second path
Separately developed check, where used
Second agent receives rules and raw cases
Sign-off
Recorded approval of a release
Explicit acceptance linked to the reviewed commit
The structure survived. The records moved into the repository. A commit identifies the version; the acceptance record names the decision and the owner.
Verification asks whether the implementation meets its requirements. Validation asks whether the result is suitable for its intended use. A passing queue test does not authorize automatic refunds.
03Experiment 1
3. Remove a control and watch the queue change
Harbor Dispatch has six input rows representing five orders. Its specified answer is A102 and A105, once each. Change the controls and compare identities as well as totals.
EXPERIMENT 1 / THE RULE
Two orders belong in this queue.
SPECIFIED ANSWER2 ordersA102 · A105
Vertical tick on each bar: 48 hours. Rust marks a decision that differs from the specified answer, whether a wrong inclusion or a wrong exclusion; green is a correct inclusion; grey is a correct exclusion. The status text names each.
Harbor Dispatchinvented teaching casesages fixed at the teaching snapshot; paid and shipped checks stay active
Inspect all six input rows
Order
Age at 4 p.m.
State
Why this case exists
A101
47 hours
Paid, unshipped
One hour too early
A102
48 hours
Paid, unshipped
Exactly at the boundary
A103
72 hours
Canceled
Canceled, despite its age
A104
60 hours
Shipped
Already shipped
A105
60 hours
Paid, unshipped
Eligible order
A105
60 hours
Paid, unshipped
Repeated copy of the same order
The same-count preset returns two copies of A105 while losing A102. Two entries still make the wrong answer. Cases chosen before implementation give the checker something concrete to disagree with.
Real evidence: a fixture exposes a missing condition
Early Warning, a Cascadia analytics build, uses a synthetic fixture to check episode construction. Its October row exceeds the forecast range but fails a separate five-report condition. Omitting that condition changes the answer. The fixture establishes behavior for those cases, not detection usefulness. Fixture · Expected result · Harness
04Experiment 2
4. Give a second agent a separate route to the answer
In this illustration, the first agent omits the cancellation rule. Checking through the same filter repeats that mistake. A second agent receives only the rules and raw cases, without the first agent's implementation or output.
EXPERIMENT 2 / THE AGENT BOUNDARY
What does the checking agent receive?
SIX RAW INPUT ROWS → two calculation paths
FIRST AGENT / BUILD
Produce the queue
CHECKING ARRANGEMENT
Original agent checks itself
In this seeded case, the separate path catches the omitted cancellation rule. Shared rules, source data and possible reasoning errors remain.
Harbor Dispatchinvented teaching casesillustrated agent behavior; no live model calls or complete-independence claim
The boundary stays: both agents share the rule and source data, and may make similar reasoning errors. Agreement supports the behavior tested; it does not establish that the rule is right, the source is true or every case is covered. The illustrated agent outcomes are fixed teaching cases, not live model calls or measured success rates.
Real evidence: a second path names its exclusions
Early Warning documents which transformations its second path recomputes. It shares exported states and points, and does not repeat candidate fitting or promotion. Its agreement has that stated boundary. Scope and exclusions
05Experiment 3
5. Make a failed check change what gets released
Detecting a problem and stopping release are separate controls. For Harbor Dispatch, the queue must match the known answer and its source must be no more than two hours old.
EXPERIMENT 3 / THE RELEASE GATE
A red check matters only if it changes the next step.
Release requires source age ≤ 2 hours.
1 hour8 hours
Harbor Dispatchinvented teaching casessimulated current release; no real system is changed
Record the failure, stop, correct and rerun. If an owner accepts an exception, retain the failed criterion and the restriction. Approval does not rewrite a failed check as a pass.
Real evidence: detection and enforcement can diverge
Early Warning's D16 records five of seven codes outside the preset coverage band. The build continued with three codes' review rules disabled as prescribed; the separate instruction to stop the build was not enforced. The owner subsequently accepted the outcome. D17 records a corrected rerun after the first run's output had been seen. Matter Ledger, another Cascadia build, had a nineteen-day stall discovered by a reader. These motivate enforced gates, retained run history and freshness monitoring; they do not establish that every control was deployed. Numbers gate · Decision record · Stall incident
06Commitments
6. Keep five commitments with the build
PROPOSED / FIVE COMMITMENTS
01Commitment
An owner-answerable questionName the use and owner; record acceptance and restrictions against a version.
02Commitment
Expected results before implementationCommit the cases and expected answers before the build being tested.
03Commitment
A separate calculation pathRe-derive the published figures; name shared inputs and the scope actually checked.
04Commitment
A check that must failInject a known defect and confirm the named check rejects it.
05Commitment
Every run and its dispositionKeep status, stop reason and resolution, including original and corrected runs.Proposed
Confidence comes from evidence that can contradict the implementation, people who review the remaining risks, and a release decision that respects both.
Across ten modules, the checks developed build by build. The Build by Build matrix links each recorded check to its evidence.
Tech stack
Static HTML, no frameworksql.js 1.14.2, SQLite in the browserFelix design system v1.0VIZ-PRINCIPLES v2.9Playwright behaviour checks
Open build
Read the checks, then the evidence.
The repository carries the manuscript, the six invented rows, both SQL queries, the behaviour checks that drive every experiment, and the chart review. The three evidence notes link to pinned commits in the Cascadia builds.