Effort is per station, and a build reads its own diff before asking you to - #126
Merged
unitypark merged 2 commits intoAug 18, 2026
Merged
Conversation
…ou to Two gaps in the agent layer, found by reading it rather than by a failure. Effort was typed and then thrown away. ModelCallOptions.effort took all five levels and defaulted to high; three call sites passed the literal 'high'; nothing configured it. Worse, the Claude Code path passed no effort at all, so in subscription mode the setting specd thought it had chosen never reached the model — `claude --effort` exists and takes the same five levels, and specd simply never sent it. One value for every station is wrong in both directions at once. xhigh is the setting for coding and agentic work and is what Claude Code itself runs at, while cheap mechanical passes are where low belongs. specd ran a build that writes code a human is about to merge at the same level as an index run that summarizes text. STATION_EFFORT now puts build and review at xhigh, spec and ground at high, index at low; effortFor() resolves it; projects.effort moves all of them, with NULL meaning no preference rather than low. The value reaches both model paths and the runner payload, so a dispatched build is not quietly cheaper than a local one. Nothing read the diff. Verify answers "do the tests pass", which the test suite already answers; nobody answered "is this the change we approved, and is it any good". ReviewAgent now reads the diff with read-only tools between verify and publish and answers in a schema — verdict, one-line summary, findings carrying path:line, a severity, and the criterion or design claim each bears on. They render into the PR body above the acceptance criteria, because the new reading belongs above what it is measured against. Four things it had to get right. It is advisory and says so in the body: a finding does not fail a build, the same line citation drift sits on. It is read-only by denial rather than omission — Write and Edit are disallowed and the pass runs under --permission-mode plan, so an over-helpful reviewer cannot amend the branch it was asked to assess. An empty findings list is a real answer and a clean pass is reported rather than omitted, because silence would be indistinguishable from a pass that never ran. And it runs where the workspace is: the prompt is authored on the API side so both paths ask the same thing, but a dispatched build reviews on the runner, whose disk holds the diff. Recorded as decisions/0021, including the cost: effort is one dial per project rather than per station, and effortFor() is the seam where that would change.
CI failed with every test passing: 506 green, one unhandled `write EPIPE` from `local-review.ts`. `run()` shells out to `gh`, `glab` and `git`, and in the common failure cases — no such binary, a rejected push — the child is gone before the prompt is written. Node reports that write as an `error` on the stdin stream rather than on the child, and with no listener it is an uncaught exception. Whether it fires at all is a scheduling race, which is why this passed on the four PRs since I introduced it. The runner already knew. `apps/runner/src/claude.ts` carries this exact guard with a comment saying CI found it there first — and the same pattern in the two API-side spawns never got it. Both now match the runner's shape rather than swallowing everything: EPIPE and ERR_STREAM_DESTROYED are the expected ones and say nothing, anything else lands in stderr, and neither crashes a process over a prompt the callee was never going to read. The regression test drives `run()` directly — the race is a property of that function, and reproducing it through `openLocalReview` would depend on which binaries the machine happens to have. Writing 8 MB to a command that exits at once loses the race every time instead of occasionally; without the guard it fails with the identical `write EPIPE` CI reported.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two gaps in the agent layer, found by reading it rather than by a failure.
1 · Effort was typed and then thrown away
ModelCallOptions.effortaccepted all five levels and defaulted tohigh.Three call sites passed the literal
'high'. Nothing configured it. And theClaude Code path passed no effort at all — so in subscription mode the
setting specd thought it had chosen never reached the model.
claude --effortexists and takes the same five levels; specd simply never sent it.
One value for every station is wrong in both directions at once.
xhighisthe documented setting for coding and agentic work — it's what Claude Code
itself runs at — while cheap mechanical passes are where
lowbelongs. specdran a build that writes code a human is about to merge at the same level as an
index run that summarises text.
high(CLI: nothing)xhighhighhighhighloweffortFor(station, project.effort)resolves it,projects.effortmoves everystation at once, and NULL means "no preference", not
low— a project thatnever chose one must not be quietly moved off the defaults. The value now
reaches the API path, both CLI paths, and the runner job payload, so a
dispatched build is not silently cheaper than a local one.
2 · Nothing read the diff
Verify answers "do the tests pass" — which the test suite already answers.
Nobody answered "is this the change we approved, and is it any good". That
reading was left entirely to whoever opened the PR, cold.
ReviewAgentnow reads the diff with read-only tools between verify andpublish, and answers in a schema: a verdict, a one-line summary, and findings
carrying
path:line, a severity, and the acceptance criterion or design claimeach bears on. They render into the PR body above the acceptance criteria —
the new reading sits above what it's measured against.
Four things it had to get right:
That's the line citation drift already sits on: an unrelated opinion must not
be able to stop an approved spec from shipping. Making findings refusable is
a house-rule decision (2.4), not a default.
WriteandEditare in--disallowed-toolsand the pass runs under--permission-mode plan, so anover-helpful reviewer cannot quietly amend the branch it was asked to assess.
rather than omitted — silence would be indistinguishable from a pass that
never ran, which is the same failure as a green verify for tests that didn't
execute. Tested.
paths ask the same thing, but a dispatched build reviews on the runner —
the diff is on the runner's disk and gone by the time the report arrives.
Building this only for the in-process path would have been the same
half-wiring as the effort bug above.
What I deliberately did not do
Replace specd's prompts with stock
/planand/designcommands. TheSpecAgent's prompts are what produce EARS criteria, citations, and the four
verdicts — swapping them for generic ones would trade the thing that makes a
spec checkable for convenience. Review was the actual gap.
Verify
pnpm typecheck && pnpm test(43 API files),pnpm build,pnpm site:check.16 new tests: the station-default invariants (including one that fails if they
ever collapse back to a single value),
effortFornull handling, and the PRrendering — severity ordering, the not-run case, the clean case, and section
order.
--effortflag values and--permission-mode planwere both checkedagainst
claude --helprather than assumed.For the reviewer
0019_project_effort.sqladds a nullable column with a CHECKconstraint. Additive; run
pnpm db:migrate.xhigh. Skipped on anempty diff, and soft everywhere — the commits exist and verify has spoken, so
a review that can't run costs an opinion, not the build. But it is a real
cost increase per build, on top of build effort rising from
hightoxhigh.and rendering are tested; the quality of what it finds is not, and won't be
until it runs on a real build.
grounding and expensive builds can't express that yet;
effortForis theseam.