Skip to content

feat(wizard): port migration rewriter to the EQL v3 domain family#771

Merged
tobyhede merged 3 commits into
remove-v2from
feat/remove-eql-v2-pr6-wizard
Jul 23, 2026
Merged

feat(wizard): port migration rewriter to the EQL v3 domain family#771
tobyhede merged 3 commits into
remove-v2from
feat/remove-eql-v2-pr6-wizard

Conversation

@tobyhede

@tobyhede tobyhede commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Part of the EQL v2 removal effort (#707). PR 6 of the linear stack — stacked on #770 (PR 5, stack-drizzle).

⚠️ Draft: stacked on #770#769#768. Review/merge PRs 3–5 first. Diff shows only wizard changes on top of PR 5.

What this does

Removes the last live EQL v2 code from @cipherstash/wizard, per the design §"PR 6".

  • Ported src/lib/rewrite-migrations.ts from the eql_v2_encrypted-only rewriter to the full eql_v3_* family (mirroring the CLI's drizzle-kit v3: ALTER COLUMN to an eql_v3 domain emits invalid DDL (no v3 rewriter) #693 fix, re-converging the two long-coupled copies). Port, not delete — it's still live: post-agent.ts calls it after drizzle-kit generate, and since the wizard now scaffolds v3, the generated ALTER COLUMN … eql_v3_* statements would otherwise slip through unrepaired and fail at migrate time. Adds near-miss flagging (RewriteResult) and a stronger data-destroying warning.
  • Domain checks flipped via a new isEqlEncryptedDomain helper (eql_v2_encrypted || startsWith('eql_v3_'), matching migrate's classifyEqlDomain convention) in wizard-tools.ts.
  • Changeset (wizard: minor). No skill change (the wizard's internal post-agent step isn't documented in any skill).

Decision 6 (deliberate)

Detection recognizes both v2 and v3 as "already encrypted" so the agent won't clobber existing v2 ciphertext, while the wizard only emits v3. No decrypt/read path removed.

Review follow-ups (all four notes addressed)

All four non-blocking notes from the approving review are now closed, in 8466300f and 9ba737ea.

  1. Hardcoded "public"."<domain>" — behaviour unchanged (it is not a regression), but the assumption is now documented at the emit site: the qualifier is asserted, not read back from the matched SQL — the schema capture is the TABLE's schema and says nothing about where the domain lives. The comment names the CLI sibling as the second place to thread an install schema through if EQL ever supports non-public domain installs. Already pinned by the existing pgSchema() test.

  2. rewriteEncryptedMigrations stopped at the first existing candidate dir — fixed, and it was a live defect, not just a latent one: an empty or already-rewritten drizzle/ sitting beside a project's real migrations/ left those migrations unrepaired, so they failed at migrate time with the very cannot cast type … error the rewriter exists to prevent. The comment justifying the early return ("running again on a different candidate would double-transform already-rewritten SQL") was wrong on both counts — distinct directories hold distinct files, and the rewrite is idempotent anyway (a rewritten statement no longer contains SET DATA TYPE, so neither matcher can fire twice). Extracted as an exported, log-free sweepMigrationDirs(cwd, dirs) so it is directly testable without mocking @clack/prompts; a directory that throws is reported via error rather than stranding the ones after it. Does not apply to the CLI sibling — it takes a single explicit outDir.

  3. skipped[].statement preamble — fixed in both copies. NEAR_MISS_RE's lazy [^;]*? has no left boundary but the previous ;, or start-of-file, so a near-miss in a file opening with a comment block was quoted back with that whole block glued to its front. Leading blank/comment lines (incl. --> statement-breakpoint) are stripped. Detection is unchanged; only the reported text differs.

  4. The 3 @cipherstash/auth AutoStrategy tsc errors — root-caused and fixed rather than deferred. @cipherstash/auth uses conditional exports (nodeindex.d.ts, which has AutoStrategy; defaultwasm-types.d.ts, which doesn't). The wizard's tsconfig sets moduleResolution: "bundler", which does not include the node condition, so tsc resolved against the wrong entry. Nothing was broken at runtime — the wizard is a Node CLI and always loads the node branch. Added "customConditions": ["node"], matching what packages/stack, packages/prisma-next, packages/test-kit, packages/stack-drizzle and packages/stack-supabase already carry for the identical protect-ffi problem.

    The reviewer's framing — "would bite a future strict type gate" — is why this also ships the gate. The wizard is built by tsup with dts: false, so the build transpiles without ever typechecking; that is exactly how these three errors sat in main unnoticed. Added a typecheck script and wired it into tests.yml beside the prisma-next gate (prisma-next: typecheck is red and not gated in CI — the type-level operator-capability tests aren't enforced #684). Verified it fires: with customConditions removed the step exits 2 on all three errors; restored, it exits 0.

Notes 2 and 3 landed with 9 regression tests, each watched failing against the pre-fix source first. Note 4 needs no changeset — dts: false means the published tarball is byte-identical.

Green gate

build ✓ · pnpm --filter @cipherstash/wizard run typecheck 0 errors (was 3) · wizard 265 pass / 5 env-skipped · stash 770 pass / 53 files · code:check 0 errors.

🤖 Generated with Claude Code

@changeset-bot

changeset-bot Bot commented Jul 23, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: d7a6123

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 11 packages
Name Type
stash Minor
@cipherstash/wizard Minor
@cipherstash/basic-example Patch
@cipherstash/e2e Patch
@cipherstash/stack Minor
@cipherstash/stack-drizzle Minor
@cipherstash/stack-supabase Minor
@cipherstash/prisma-next Minor
@cipherstash/bench Patch
@cipherstash/test-kit Patch
@cipherstash/prisma-next-example Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: fab616d5-f355-4e69-9118-48cba120ab22

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/remove-eql-v2-pr6-wizard

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@tobyhede
tobyhede force-pushed the feat/remove-eql-v2-pr5-drizzle branch from 81ed000 to a1bfe1a Compare July 23, 2026 01:26
@tobyhede
tobyhede force-pushed the feat/remove-eql-v2-pr6-wizard branch from bdabbac to 80e93e1 Compare July 23, 2026 01:26
@tobyhede
tobyhede force-pushed the feat/remove-eql-v2-pr5-drizzle branch from a1bfe1a to 36b099b Compare July 23, 2026 02:07
@tobyhede
tobyhede force-pushed the feat/remove-eql-v2-pr6-wizard branch 2 times, most recently from 27c028f to 3e88398 Compare July 23, 2026 02:11
@tobyhede
tobyhede force-pushed the feat/remove-eql-v2-pr5-drizzle branch from 2a1dca6 to 30e7b55 Compare July 23, 2026 02:17
@tobyhede
tobyhede force-pushed the feat/remove-eql-v2-pr6-wizard branch from 3e88398 to cd75ef4 Compare July 23, 2026 02:18
@tobyhede
tobyhede marked this pull request as ready for review July 23, 2026 03:27
@tobyhede
tobyhede requested a review from a team as a code owner July 23, 2026 03:27

@freshtonic freshtonic left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed by Claude Code on behalf of James Sadler.

Verdict: Approve

This is a clean, well-reasoned port of the wizard's migration rewriter from the single eql_v2_encrypted type to the full eql_v2/eql_v3_* domain family, re-converging it with the CLI sibling (packages/cli/src/commands/db/rewrite-migrations.ts) landed in #693. I fetched both files at the PR head and confirmed the regex machinery is now identical to the CLI reference. The change is a detection/repair transform, not a crypto change, and I found no blocking correctness, security, or data-loss defects.

What I checked carefully

  • Regex family (ENCRYPTED_DOMAINDOMAIN_RE / MANGLED_TYPE_FORMS / ALTER_COLUMN_TO_ENCRYPTED_RE). Derived-from-one-source design (DOMAIN_RE built from ENCRYPTED_DOMAIN) is good — it structurally prevents matcher/extractor drift, and the it.each(V3_DOMAINS) "extracts the bare domain" test enforces it. Longest-first alternation ordering is correct; I traced the plain, public., "public"."...", "undefined"."...", and double-quoted forms and none premature-match a prefix and strand a fragment before \s*;. Capture-group indexing in the replace callback (first/second/column/mangledType) is right because every MANGLED_TYPE_FORMS branch uses non-capturing groups.
  • The \s*; tightening. Switching the tail from [^;]*; to (...)\s*; is the load-bearing safety fix: a hand-authored SET DATA TYPE ... USING <expr>; no longer strict-matches (its USING would otherwise have been silently discarded). Correctly backstopped by NEAR_MISS_RE, which flags-but-does-not-rewrite. Verified by the 0013_using test.
  • NEAR_MISS_RE ordering. Running the broad scan on the post-rewrite content is correct: genuinely-rewritten statements no longer contain SET DATA TYPE, so they can't self-flag. The "reports no skipped statements when the strict rewrite fully handled the file" test pins this, and the emitted guidance/-- UPDATE ... SET ... comment lines contain no SET DATA TYPE, so they don't trip it either. [^;] bounding keeps matches within a single statement even across newlines.
  • Data-loss surface. ADD+DROP+RENAME is data-destroying — but that's unchanged from the prior v2 rewriter (deliberate, per design Decision 6). The new header comment and the post-agent.ts warning are a genuine improvement over the old "backfill ... Empty tables are safe as-is" text, which understated the footgun. The added note that constraints/defaults/indexes are not carried over is a good catch.
  • Detection (isEqlEncryptedDomain). === 'eql_v2_encrypted' || startsWith('eql_v3_') errs conservative (treats-as-encrypted), so the failure mode is "won't scaffold over a column" rather than "clobbers ciphertext" — the safe direction. Trailing-underscore prefix correctly avoids a hypothetical eql_v30_*.
  • post-agent.ts wiring. Destructures the new RewriteResult, surfaces skipped non-fatally, per-dir try/catch. Fine.

Non-blocking notes (no change required)

  1. Hardcoded "public"."<domain>" in the emitted ADD COLUMN. The rewrite always targets the EQL domain in public, regardless of where it's actually installed. This matches both the prior v2 code and the CLI sibling, and public is the EQL install convention, so it's not a regression — just a latent assumption worth keeping in mind if EQL ever supports non-public domain installs.
  2. rewriteEncryptedMigrations returns after the first existing candidate dir, even if that dir contained zero matches, so a later candidate dir with real EQL alters would be skipped. Pre-existing behavior, not introduced here — flagging only for awareness.
  3. skipped[].statement can include leading content back to the previous ; (e.g. a leading comment block at file top) because NEAR_MISS_RE starts with lazy [^;]*?. Cosmetic only — the warning is still actionable.
  4. PR notes 3 pre-existing @cipherstash/auth AutoStrategy tsc errors in untouched files; agreed those are out of scope here but would bite a future strict type gate.

Test coverage is strong (108 new tests, including the full generated v3 domain cross-product and every mangled drizzle-kit form). Good work.

tobyhede added a commit that referenced this pull request Jul 23, 2026
…ypecheck

Closes the last open item from the #771 review — the 3 pre-existing
`AutoStrategy` tsc errors flagged as "would bite a future strict type gate".

**Root cause.** `@cipherstash/auth` uses conditional exports: `node` resolves
to `index.d.ts` (the full surface, including `AutoStrategy`), `default`
resolves to `wasm-types.d.ts` (which has no `AutoStrategy`). The wizard's
tsconfig sets `moduleResolution: "bundler"`, which does NOT include the `node`
condition, so tsc took the `default` branch and reported `AutoStrategy` as
missing on all three call sites — `agent/fetch-prompt.ts`, `agent/interface.ts`
and `lib/prerequisites.ts`.

Nothing was actually broken at runtime: the wizard is a Node CLI and always
loads the `node` branch. The types were simply resolved against the wrong
entry point.

**Fix.** Add `"customConditions": ["node"]`, matching what `packages/stack`,
`packages/prisma-next`, `packages/test-kit`, `packages/stack-drizzle` and
`packages/stack-supabase` already carry for the identical `protect-ffi`
conditional-export problem. `tsc --noEmit` now exits 0.

**Regression guard.** A type-level defect needs a type-level gate, or it comes
back silently — the wizard is built by tsup with `dts: false`, so the build
transpiles without ever typechecking, which is exactly why these three errors
sat in `main` unnoticed. Add a `typecheck` script and wire it into `tests.yml`
alongside the prisma-next gate (#684). Verified the gate fires: with
`customConditions` removed the step exits 2 on all three errors; restored, it
exits 0.

No changeset — `dts: false` means the published tarball is byte-identical.
Tooling only, no observable behaviour change.

Green: wizard 265 pass / 5 env-skipped, typecheck 0 errors; stash 770 pass /
53 files; `code:check` 0 errors.
@tobyhede

Copy link
Copy Markdown
Contributor Author

All four non-blocking notes are addressed — 8466300f (notes 1–3) and 9ba737ea (note 4).

1 — hardcoded "public"."<domain>". Left as-is, documented at the emit site. Worth stating explicitly: the qualifier is asserted, not read back from the matched SQL — the schema capture is the TABLE's schema (from a pgSchema() table) and says nothing about where the domain lives. The comment names the CLI sibling as the second place to thread an install schema through if that ever changes.

2 — sweep stopping at the first existing candidate dir. Fixed. This turned out to be live rather than latent: an empty or already-rewritten drizzle/ beside a project's real migrations/ left those migrations unrepaired, so they failed at migrate time with the exact cannot cast type … error the rewriter exists to prevent. The comment justifying the early return ("running again on a different candidate would double-transform already-rewritten SQL") was wrong twice over — distinct directories hold distinct files, and the rewrite is idempotent regardless, since a rewritten statement no longer contains SET DATA TYPE. Extracted as an exported, log-free sweepMigrationDirs(cwd, dirs) so it is testable without mocking @clack/prompts; a directory that throws is reported via error instead of stranding the ones after it. Not applicable to the CLI sibling — it takes a single explicit outDir.

3 — skipped[].statement preamble. Fixed in both copies. Leading blank/comment lines (incl. --> statement-breakpoint) are stripped. Detection unchanged; only the reported text differs.

4 — the 3 AutoStrategy tsc errors. Root-caused and fixed rather than deferred. @cipherstash/auth uses conditional exports — nodeindex.d.ts (has AutoStrategy), defaultwasm-types.d.ts (doesn't). The wizard's tsconfig sets moduleResolution: "bundler", which does not include the node condition, so tsc resolved against the wrong entry. Nothing was broken at runtime; the wizard is a Node CLI and always loads the node branch. Added "customConditions": ["node"], matching what stack, prisma-next, test-kit, stack-drizzle and stack-supabase already carry for the identical protect-ffi problem.

Your "would bite a future strict type gate" framing is why this also ships the gate. The wizard is built by tsup with dts: false, so the build transpiles without ever typechecking — which is precisely how these three sat in main unnoticed. Added a typecheck script wired into tests.yml beside the prisma-next gate (#684), and verified it fires: with customConditions removed the step exits 2 on all three errors; restored, exits 0.

Notes 2 and 3 landed with 9 regression tests, each watched failing against the pre-fix source before the fix went in. Note 4 needs no changeset — dts: false means the published tarball is byte-identical.

Green: typecheck 0 errors (was 3) · wizard 265 pass / 5 env-skipped · stash 770 pass / 53 files · code:check 0 errors.

Base automatically changed from feat/remove-eql-v2-pr5-drizzle to remove-v2 July 23, 2026 08:17
tobyhede added 3 commits July 23, 2026 18:19
The wizard now scaffolds EQL v3 columns, so `drizzle-kit generate` emits
`ALTER COLUMN ... SET DATA TYPE eql_v3_<name>` — which Postgres rejects (no
cast from text/numeric to an EQL domain). The post-agent rewriter matched only
the single `eql_v2_encrypted` type, so those v3 statements slipped through
unrepaired and failed at migrate time.

Port the rewriter to the whole `eql_v3_*` concrete-domain family alongside
legacy `eql_v2_encrypted`, mirroring the sibling CLI fix (#693): every mangled
form drizzle-kit emits (incl. the 0.31.0+ `"undefined".` prefix and
schema-qualified pgSchema tables), near-miss flagging for `SET DATA TYPE ...
USING ...` it cannot safely repair, statement-breakpoints, and a clearer
data-destroying / empty-table-only warning that points populated tables at the
staged `stash encrypt` flow.

Database introspection (`isEqlEncrypted`) now recognises BOTH `eql_v2_encrypted`
and the `eql_v3_*` family as already-encrypted, matching migrate's
`classifyEqlDomain` v3 convention — so the agent won't scaffold over existing
encrypted data of either generation (v2 ciphertext stays valid and detected).

Add a rewrite-migrations test suite (adapted from the CLI's).
Address code-review feedback on the v3 migration-rewriter port.

**Sweep every candidate migration directory.** `rewriteEncryptedMigrations`
returned after the FIRST candidate that merely existed, even when that
directory contained zero matches — so an empty or already-rewritten
`drizzle/` sitting next to a project's real `migrations/` left those
migrations unrepaired, and they then failed at migrate time with the very
`cannot cast type ...` error the rewriter exists to prevent.

The comment justifying the early return ("running again on a different
candidate would double-transform already-rewritten SQL") was wrong on both
counts: distinct directories hold distinct files, and the rewrite is
idempotent anyway — a rewritten statement no longer contains `SET DATA
TYPE`, so neither the strict matcher nor the near-miss scan can match it a
second time.

Extract the sweep into an exported, log-free `sweepMigrationDirs(cwd, dirs)`
so it is directly testable without mocking @clack/prompts. It sweeps every
existing candidate and reports a directory that throws via `error` rather
than stranding the ones after it; post-agent keeps the reporting.

**Trim the near-miss statement preamble.** `NEAR_MISS_RE` opens with a lazy
`[^;]*?` whose only left boundary is the previous `;` — or the start of file
when there is none. The reported statement therefore dragged in every
comment and blank line since then, so a near-miss in a file opening with a
comment block was quoted back to the user with the whole header glued to its
front. Strip leading blank/comment lines (incl. `--> statement-breakpoint`)
so the statement reads as the offending statement alone. Detection is
unchanged; only the text shown to the user differs.

Applied to the `stash` CLI sibling too — it carried the identical defect and
its header mandates keeping the two in sync. The directory sweep does not
apply there: the CLI takes a single explicit `outDir`.

**Document the hardcoded `"public"."<domain>"`.** Behaviour unchanged and
not a regression, but worth stating: the domain qualifier is an assumption
(EQL installs into `public`), not something read back from the matched SQL —
the `schema` capture is the TABLE's schema and says nothing about where the
domain lives. Non-public domain installs would need it threaded in here and
in the CLI sibling. Already pinned by the existing pgSchema() test.

9 new tests across the two packages, each watched failing first.
…ypecheck

Closes the last open item from the #771 review — the 3 pre-existing
`AutoStrategy` tsc errors flagged as "would bite a future strict type gate".

**Root cause.** `@cipherstash/auth` uses conditional exports: `node` resolves
to `index.d.ts` (the full surface, including `AutoStrategy`), `default`
resolves to `wasm-types.d.ts` (which has no `AutoStrategy`). The wizard's
tsconfig sets `moduleResolution: "bundler"`, which does NOT include the `node`
condition, so tsc took the `default` branch and reported `AutoStrategy` as
missing on all three call sites — `agent/fetch-prompt.ts`, `agent/interface.ts`
and `lib/prerequisites.ts`.

Nothing was actually broken at runtime: the wizard is a Node CLI and always
loads the `node` branch. The types were simply resolved against the wrong
entry point.

**Fix.** Add `"customConditions": ["node"]`, matching what `packages/stack`,
`packages/prisma-next`, `packages/test-kit`, `packages/stack-drizzle` and
`packages/stack-supabase` already carry for the identical `protect-ffi`
conditional-export problem. `tsc --noEmit` now exits 0.

**Regression guard.** A type-level defect needs a type-level gate, or it comes
back silently — the wizard is built by tsup with `dts: false`, so the build
transpiles without ever typechecking, which is exactly why these three errors
sat in `main` unnoticed. Add a `typecheck` script and wire it into `tests.yml`
alongside the prisma-next gate (#684). Verified the gate fires: with
`customConditions` removed the step exits 2 on all three errors; restored, it
exits 0.

No changeset — `dts: false` means the published tarball is byte-identical.
Tooling only, no observable behaviour change.

Green: wizard 265 pass / 5 env-skipped, typecheck 0 errors; stash 770 pass /
53 files; `code:check` 0 errors.
@tobyhede
tobyhede force-pushed the feat/remove-eql-v2-pr6-wizard branch from 9ba737e to d7a6123 Compare July 23, 2026 08:20
@tobyhede
tobyhede merged commit 38c0101 into remove-v2 Jul 23, 2026
8 checks passed
@tobyhede
tobyhede deleted the feat/remove-eql-v2-pr6-wizard branch July 23, 2026 08:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants