Skip to content

plan(v0.58): add RQ-58-SHIPVERIFY (#1000) — we ship a verifier nobody can run - #1001

Merged
avrabe merged 1 commit into
mainfrom
plan/v058-shipverify
Aug 19, 2026
Merged

plan(v0.58): add RQ-58-SHIPVERIFY (#1000) — we ship a verifier nobody can run#1001
avrabe merged 1 commit into
mainfrom
plan/v058-shipverify

Conversation

@avrabe

@avrabe avrabe commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Adds RQ-58-SHIPVERIFY for #1000, and corrects the stale README sentence that kept the issue's escape hatch alive.

The problem

synth verify exists in the tree and in --help, and fails closed with an exemplary message — but it is in no artifact a user can obtain. The reporter confirmed it on the layer build (0.55.0) and the current release (0.57.0), whose four platform tarballs all lack the feature.

For a compiler whose entire argument is verified code generation, the ASIL-D translation-validation path being unobtainable is the gap worth closing first. It is also uncomfortably on-theme for v0.58: we ship a verifier nobody can run.

The escape hatch is dead — measured, not assumed

#1000 offers "or, if the Z3 dependency makes that undesirable for the default artifact, publish a variant". There is no Z3 dependency:

build z3 nodes
cargo tree -p synth-cli --features verify 0
... --features verify,synth-verify/z3-solver 2

verify = ["synth-verify"] and nothing more. z3-solver = ["z3"] is a separate opt-in with optional = true, and even enabled it links the system libz3 rather than bundling it (#553). Ordeal (pure Rust QF_BV) has been the default engine since v0.27.0.

So shipping --features verify costs no Z3, no libz3, no C++ toolchain — a release-workflow change, not a trade-off.

README:98 is the other half of the finding

It said the CLI verify feature "currently also enables the feature-gated Z3 differential oracle (statically linked)". False since v0.27.0 — and precisely the sentence that would make a maintainer accept the escape hatch and not ship the feature.

Prose that was true once, quietly stopped being, and stayed load-bearing on a release decision. Same class the v0.57 cold review kept surfacing; found this time by a user questioning a stale premise. Corrected here.

Scope, with the weight in the middle

  1. Build the released binaries (four tarballs + npm) with --features verify.
  2. A non-vacuity gate — a released-artifact smoke test that actually runs synth verify on a freshly compiled module and asserts a verdict, not the capability-missing error. Red-first against a binary built without the feature. Without it, "we shipped the feature" regresses silently to "we shipped the help text".
  3. Move the capability check ahead of the banner — today Strategy: Per-rule SMT verification (ASIL D path) prints before the tool discovers it cannot verify, so a log-scraper finds the ASIL-D line in a run that verified nothing. Exit code is already correct.

Out of scope, named: #1000's synthesize-vs-compile overlap, --format json, help-line width.

Also re-grades RQ-58-MIRRORS (#993) and RQ-58-SELECT973 (#992) → implemented; both merged and the ledger had not caught up.

Gates

rivet 50 errors before and after (unchanged); warnings +2, the standard pair every artifact carries. claim_check 49/49.

Refs #1000, #242

… can run

#1000, from a toolchain-wide CLI survey: `synth verify` exists in the tree and
in `--help`, fails closed with an exemplary message — and is in NO artifact a
user can obtain. Reproduced by the reporter on the layer build (0.55.0) AND the
current release (0.57.0), whose four platform tarballs all lack the feature.

THE ISSUE'S OWN ESCAPE HATCH IS DEAD, AND I MEASURED IT RATHER THAN ASSUMING.

#1000 offers: "or, if the Z3 dependency makes that undesirable for the default
artifact, publish a variant". There is no Z3 dependency:

  cargo tree -p synth-cli --features verify                    ->  0 z3 nodes
  cargo tree -p synth-cli --features verify,…/z3-solver        ->  2 z3 nodes

`verify = ["synth-verify"]` and nothing more. `z3-solver = ["z3"]` is a separate
opt-in with `optional = true`, and even enabled it links the SYSTEM libz3 rather
than bundling it (#553). Ordeal (pure Rust QF_BV) has been the default engine
since v0.27.0. Shipping `--features verify` costs no Z3, no libz3, no C++
toolchain — so this is a release-workflow change, not a trade-off.

README.md:98 IS THE OTHER HALF OF THE FINDING, and it is corrected here. It
said the CLI `verify` feature "currently also enables the feature-gated Z3
differential oracle (statically linked)". False since v0.27.0 — and it is
exactly the sentence that would make a maintainer accept #1000's escape hatch
and NOT ship the feature. Prose that was true once, quietly stopped being, and
stayed load-bearing on a release decision: the class the v0.57 cold review kept
turning up, found again by a user asking a good question about a stale premise.

Artifact scope, three parts, with the middle one carrying the weight:
  1. build the released binaries (four tarballs + npm) with `--features verify`
  2. a NON-VACUITY GATE — a released-artifact smoke test that actually RUNS
     `synth verify` on a freshly compiled module and asserts a VERDICT, not the
     capability-missing error. Red-first against a binary built without the
     feature. Without this, "we shipped the feature" regresses silently to "we
     shipped the help text", which is the class this whole release closes.
  3. move the capability check AHEAD of the banner — today the four
     `Translation validation:` lines including `Strategy: Per-rule SMT
     verification (ASIL D path)` print BEFORE the tool discovers it cannot
     verify, so a log-scraper finds the ASIL-D line in a run that verified
     nothing. Exit code is already correct.

Out of scope, named not dropped: #1000's `synthesize`-vs-`compile` description
overlap, `--format json`, help-line width.

Also re-graded RQ-58-MIRRORS (#993) and RQ-58-SELECT973 (#992) proposed ->
implemented; both merged and the ledger had not caught up.

rivet: 50 errors before AND after (unchanged); warnings +2, the standard pair
every artifact carries. claim_check 49/49.

Refs #1000, #242

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJK5LZZEkV5smCY1jKn18L
@codecov

codecov Bot commented Aug 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@avrabe
avrabe merged commit 93593ed into main Aug 19, 2026
107 of 110 checks passed
@avrabe
avrabe deleted the plan/v058-shipverify branch August 19, 2026 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant