The first post about my workflow came out on November 9. Five days later, I decided to measure whether that workflow actually worked. It did, but the filter that found the most problems only ran on some of the tasks.
This is v2. The core idea is the same: Claude Opus 5.5 plans with me, GPT agents execute in isolated worktrees, and nothing lands on the branch without review. Almost everything else changed, and now I have numbers on what that stopped.
344
tasks delegated in v2, Nov 14 to 22
646
errors that would have gotten through, at most
≈274
would show up in normal use (range: 176 to 374)
514
spec errors stopped before any code existed
What motivated the change
On November 14, I audited 30 days of my Claude Code usage: 188 sessions, 356 cumulative active hours and the records of every delegation. Three things stood out.
The filter that found the most problems only ran on some of the tasks. The reviewer requested changes in 56 of 98 reviews (57%), with a median of 32 seconds each. Even so, the rule kept it for large diffs or sensitive areas.
The spec was blocking the executor. In 79 of 493 reports (16%), the executor came back with UNSURE: because of a contradiction or gap in the spec itself.
The same kinds of failure kept coming back. In one large project, there were 61 continue runs per 100 tasks, against 40 on average in the others. About 47% fixed a reviewer finding, nearly always edge-case logic: races, remounting components, stale data, queue limits. In several cases, the tests passed without catching the bug. Another 20% were tasks abandoned halfway, and 30% were decisions the spec hadn't covered.
So the problem wasn't only the model. It was the model, weak tests and holes in the spec.
The second motivation came a week later, and that one was on me. Measuring the sessions, from November 14 to 17 I wrote 100 to 360 words per design, with decisions that changed what got built. From the 19th to the 21st, it was 5 to 18 words: "let's go with A", "go ahead", "I'll take the suggestion". The orchestrator arrived with a recommendation and I confirmed it. This after I had said, days earlier, that my main focus is the planning stage.
On top of that, the orchestrator was spending its context reading whole diffs, when its job is something else. And we weren't parallelizing much: of 76 recorded rounds, 45 had a single task.
What changed
| First post (Nov 9) | v2 (Nov 22) | |
|---|---|---|
| Executor | GPT-6.1 Terra, xhigh | GPT-6.2 Sol, high |
| Investigation | GPT-6.1 Sol, high | GPT-6.2 Astra, high |
| Front-end | the executor | frontend-engineer: Sol 6.2 with a UI prompt |
| Queries | the executor | scout: Terra 6.1, read-only |
| Code review | GPT-6 Astra, relevant diffs only | GPT-6.2 Astra, every task, three layers |
| Plan review | didn't exist | Claude Opus 5.5, xhigh, every round |
| Spec | no edge cases | required Edge cases and a Bite check |
| Execution | parallel when it was obvious | gated waves, up to 4 concurrent tasks |
| Orchestrator | reads the diff and fixes small things | reads reports and writes no code |
| Entry point | I drove each step | /implement, with three stops for me |
How many errors would have gotten through
This is the question I wanted to answer: if I had accepted what the executor delivered as done, how many errors would have reached the code?
None of these errors were reproduced. The numbers come from the reviewers' reports in the sessions and in each delegation's artifacts, 666 reviews in v2 alone. I split them by workflow phase:
| Phase | Tasks | Flagged | Would pass | Normal use | Confirmed | Spec |
|---|---|---|---|---|---|---|
| Nov 1–7: no routine reviewer | 173 | 1 | n/a | n/a | 0 | 0 |
| Nov 7–14: the first post's workflow | 184 | 88 | 83 | 35 | 8 | 1 |
| Nov 14–17: automatic and plan review | 161 | 247 | 175 | 74 | 99 | 318 |
| Nov 17–21: GPT-6.2, three layers, bite check | 130 | 489 | 375 | 159 | 131 | 157 |
| Nov 21–22: waves, gates and reports | 53 | 135 | 96 | 41 | 50 | 39 |
| v2 (last three phases) | 344 | 871 | 646 | 274 | 280 | 514 |
| Total | 701 | 960 | 729 | 309 | 288 | 515 |
Each column counts something different (n/a means not measured):
- Flagged:
blockerormajorfindings about logic or an unmet criterion, with repeats grouped. Style, scope and test quality are left out. - Would pass: the flagged errors already present in the first review, meaning in the delivery the executor called done. Those that appeared during fixes wouldn't exist in direct development.
- Normal use: I read a sample of 40 errors and classified each one. 17 would show up in normal use, 21 depend on a rare condition and 2 are cosmetic or unlikely. The column applies that 17-in-40 ratio.
- Confirmed: first-review errors that a re-review of the same delegation did not flag again. This is the most solid floor.
- Spec: serious plan-review findings, before any code existed. They stay out of "would pass", because not all of them would have become bugs.
The columns are not a funnel. "Normal use" is a sample-based estimate and "confirmed" is a direct count, which is why the two land close together without one containing the other.
- Flagged
- Would have passed
- Confirmed
Number of errors
The last phase has about one day of use. The jump in Nov 17–21 comes largely from the new reviewer, which finds more problems in the same diffs.
To make it concrete, here are three errors from the sample that would show up in normal use, all fixed before landing:
- The edit that vanishes. You save a form and start another edit while the refresh is still pending. When it returns, the second edit disappears and the screen says there's nothing to save.
- One Esc that closes two things. Typing
@in the channel input and pressing Esc closed the mention menu and also the members panel, and saved the panel as closed. - A judge outage counted as a model error. In a benchmark, a 503 from the service that grades answers became a negative judgment and dragged down the tested model's success rate.
What these numbers don't say
- 646 is likely a ceiling. In direct development, CI, tests or I would catch some of these errors, and whoever wrote the code might make different ones.
- The phases don't compare quality with each other. The reviewer changed along the way. On the same diffs, Astra 6.2 found more problems than Astra 6, which explains much of the jump in the Nov 17–21 phase.
- The sample is small. It's 40 items classified in one read, with a margin of about 15 percentage points. Hence the 176 to 374 range.
- The grouping is approximate. The same error reworded can count twice, and two similar errors can become one.
- The first phase is undercounted. There was almost no reviewer, so the errors in those 173 tasks weren't measured.
The cost
Reviewing every task shows up in the same data, on the other side.
continue runs per run.sh new, across all projects
Part of this is the workflow doing its job: each finding now goes back to the executor as a continue. In the current phase, 83 of 91 reviews requested changes. But my goal was for this number to drop, and so far it has gone up. I can't yet separate how much is a stricter reviewer and how much is the executor delivering worse work.
How it works now
Everything starts with /implement and a card. It keeps the task's state, resumes where it left off and only calls me at three points: design, plan alignment and delivery.
card ─► triage ─┬─ fix ─────► short design
└─ feature ─► I describe it, then the design
│ stop 1
▼
wave plan ─► plan review
│ contradiction? stop 2
▼
wave 0 (contract) ─► gate ─► wave 1 (up to 4) ─► gate
│
▼
final review ─► delivery (stop 3)I went back to planning
The card now gets classified first. A fix keeps the short design and a single approval. On a feature, the orchestrator can't show up with the answer: it first asks me open questions, with no options attached, about the problem, what success looks like and what would be a failure even if it worked. Only then do the approaches come.
The plan can also come back to me, but only in three cases: a contradiction that forces a choice between two behaviors, a design gap that changes what the user sees or the architecture, or a new contract the design didn't describe. Otherwise, it moves on. An alignment that only asks for an "ok" counts as a workflow failure.
Review before the code
Before any delegation, a Claude Opus 5.5 at xhigh reads the design, the plan and every spec in the round, looking for contradictions, open behavior, verification that can't run and parallel tasks touching the same file. That's where the 514 spec errors come from.
The spec got stricter too. Edge cases is required, with input and expected result, and run.sh refuses a spec without it.
Waves
A round is split into waves. The next wave only starts once the previous one is integrated and verified.
Wave 0 · contract
team-channel-contract fast-executor core/src/types.rs, ui/src/types.ts
Wave 1 · depends on wave 0
channel-backend fast-executor core/src/channel/
channel-ui frontend-engineer ui/src/channel/
mention-parser fast-executor core/src/mention.rs
Wave 2 · depends on wave 1
channel-e2e frontend-engineer ui/e2e/channel.spec.tsThe rules are few:
- One owner per file. Two tasks in the same wave never touch the same path.
- Hotspot files (routes, registries, config, lockfiles, i18n catalogs) only change in wave 0 or in an integration task.
- Wave 0 only exists when there's a contract: types, interfaces, schema and routes, with no logic.
- At most 4 active tasks per session.
run.shrefuses the fifth. - A gate between waves. Tests, build and typecheck on the integrated branch, plus a review of the wave to catch the seams between tasks that no individual reviewer sees.
Three-layer review and the test that bites
Every diff is reviewed automatically, in a single pass:
- Completeness: each acceptance criterion becomes met, not met or unverifiable, with evidence.
- Logic and edge cases: races, remounts, stale data, limits, error paths.
- Tests and scope: would any test pass without the change? Does each part of the diff map to an item in the spec?
Before the reviewer, run.sh runs mechanical checks: disabled tests, new TODOs, files outside the scope, files from a neighboring task. And when the spec has a Bite check, it copies the worktree, reverts only the production code and runs the tests. If they still pass, they prove nothing, and that becomes a finding.
The orchestrator stopped reading diffs
Reading diffs polluted the context of the one who needs to plan, and that gets worse with four tasks per wave. Now the orchestrator reads the verdict, the findings, the checks and the executor's report. Code findings go straight back to the executor. If the executor disagrees, it answers with DISPUTE: and Astra arbitrates.
The risk is the reviewer becoming the only barrier. In the comparison test, Astra 6 approved a task with a real cancellation bug, which 6.2 caught. That's why there are three exceptions where the orchestrator opens that task's diff: a sensitive area, a red gate with everything approved, and the same finding coming back after two continue runs.
What I tested on the models
Before settling the roles, I ran four real tasks that had findings at the time, from the same commit and with the original spec:
| Executor | Blocker/major | Unmet criteria | Abandoned items |
|---|---|---|---|
| GPT-6.1 Terra, xhigh | 14 | 17 | 4 |
| GPT-6.2 Sol, medium | 11 | 9 | 0 |
| GPT-6.2 Sol, high | 12 | 12 | 0 |
Terra abandoned items halfway and Sol didn't. Between medium and high, the difference was within the noise, and I stayed with high because quality comes first. It's one run per combination, so treat it as an indication rather than a benchmark.
I also tested Claude Sonnet 5.5 on the front-end. On the two UI tasks, it did worse on one and tied on the other, took about twice as long and produced larger diffs. Today the front-end runs on Sol 6.2, with its own UI prompt.
What's still open
- Waves have about one day of use. They're still being validated, and the next real feature is the true test.
continueruns per task need to drop. If they don't, the next step is a second reviewer from another model family.- It needs remeasuring with more time on v2. The current phase is still running, and its numbers will change.