Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .changeset/remove-eql-v2-drizzle-root.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
'@cipherstash/stack-drizzle': major
'@cipherstash/stack': patch
'stash': patch
---

Remove the EQL v2 authoring surface from `@cipherstash/stack-drizzle` and collapse the EQL v3 `./v3` subpath into the package root.

**Breaking (`@cipherstash/stack-drizzle`):**

- The EQL v2 root exports are gone: `encryptedType`, the v2 `extractEncryptionSchema`, the v2 `createEncryptionOperators` (including the `like` / `ilike` operators), and `EncryptionConfigError`. Authoring or querying `eql_v2_encrypted` columns through Drizzle is no longer supported.
- The `./v3` subpath is **removed** from the package `exports` map and `typesVersions`. The EQL v3 implementation is now the package root, and the `*V3` names are de-suffixed (`createEncryptionOperatorsV3` → `createEncryptionOperators`, `extractEncryptionSchemaV3` → `extractEncryptionSchema`). This is a **hard break with no alias**: post-collapse the root names would collide with the removed v2 names, and keeping an alias would silently type-check v2 call sites against v3 semantics.

**Migration** — import the v3 surface from the package root instead of `./v3`, and drop the `V3` suffix:

```diff
- import { types, extractEncryptionSchemaV3, createEncryptionOperatorsV3 } from '@cipherstash/stack-drizzle/v3'
+ import { types, extractEncryptionSchema, createEncryptionOperators } from '@cipherstash/stack-drizzle'
```

The `types.*` column factories, `makeEqlV3Column` / `getEqlV3Column` / `isEqlV3Column`, the codec helpers (`v3ToDriver` / `v3FromDriver` / `EqlV3CodecError`), and `EncryptionOperatorError` are unchanged apart from moving to the root.

Existing EQL v2 ciphertext remains decryptable via `@cipherstash/stack` — only the Drizzle-side v2 authoring and query-building is removed.

**`stash` (patch):** `stash init --drizzle` scaffolded the removed surface — the generated encryption-client file and the Drizzle placeholder both imported `extractEncryptionSchemaV3` from `@cipherstash/stack-drizzle/v3`, so a freshly-initialised project would not resolve against this release. Both now emit the collapsed root import. The bundled `stash-drizzle` and `stash-encryption` skills are updated to match.

**`@cipherstash/stack` (patch):** README only — its Drizzle section documented the removed `./v3` subpath and the `*V3` export names.
53 changes: 32 additions & 21 deletions .github/workflows/tests-bench.yml
Original file line number Diff line number Diff line change
@@ -1,20 +1,37 @@
name: "Test Benchmark JS"

# Trigger paths follow the package GRAPH, not the bench directory. bench depends
# on `@cipherstash/stack` and `@cipherstash/stack-drizzle`, and (since the fixture
# schema moved to EQL v3 domains) on the CLI's EQL installer — a break in any of
# them can red this job, so an edit to any of them has to be able to trigger it.
# Scoped the same way the `integration-*` workflows scope theirs rather than
# taking whole packages, to keep the trigger surface honest.
on:
push:
branches:
- 'main'
paths:
- 'packages/bench/**'
- 'packages/stack/src/eql/v3/**'
- 'packages/stack-drizzle/**'
- 'packages/test-kit/**'
- 'packages/cli/src/installer/**'
- 'local/**'
- '.github/workflows/tests-bench.yml'
- '.github/actions/integration-setup/**'
pull_request:
branches:
- "**"
# Repeated verbatim: GitHub Actions does not support YAML anchors/aliases.
paths:
- 'packages/bench/**'
- 'packages/stack/src/eql/v3/**'
- 'packages/stack-drizzle/**'
- 'packages/test-kit/**'
- 'packages/cli/src/installer/**'
- 'local/**'
- '.github/workflows/tests-bench.yml'
- '.github/actions/integration-setup/**'

jobs:
tests-bench:
Expand All @@ -25,25 +42,14 @@ jobs:
- name: Checkout Repo
uses: actions/checkout@v6

- uses: pnpm/action-setup@v6.0.9
name: Install pnpm
with:
run_install: false

- name: Install Node.js
uses: actions/setup-node@v6.4.0
# The shared integration preamble, which also builds the `stash` CLI — not
# incidental here: the bench `globalSetup` installs EQL v3 by shelling out
# to the real `stash eql install --eql-version 3`.
- uses: ./.github/actions/integration-setup
with:
# This job's existing Node; the composite defaults to 24 for the
# integration suites.
node-version: 22
cache: 'pnpm'

# node-pty's install hook falls back to `node-gyp rebuild` when no
# linux-x64 prebuild matches. pnpm/action-setup v6 no longer ships
# node-gyp on PATH, so install it explicitly.
- name: Install node-gyp
run: npm install -g node-gyp

- name: Install dependencies
run: pnpm recursive install --frozen-lockfile

# `@cipherstash/stack` ships dist/-based `exports`; bench imports
# from `@cipherstash/stack` and `@cipherstash/stack-drizzle` (the Drizzle
Expand All @@ -54,11 +60,16 @@ jobs:
- name: Build stack + adapter packages
run: pnpm exec turbo run build --filter @cipherstash/stack --filter @cipherstash/stack-drizzle

# Starts the pinned postgres-eql container (PostgreSQL 17 + EQL
# pre-installed) via local/docker-compose.yml; waits for healthcheck.
- name: Start local Postgres (EQL)
# Service-scoped `postgres`: local/docker-compose.yml also defines a
# PostgREST that bench never talks to.
#
# This container has EQL v2 baked in and NOTHING to do with the v3 domains
# the fixture schema needs — those come from the `globalSetup` install
# below. Do not add an EQL step here; keeping the install inside vitest is
# what makes `pnpm test:local` and CI the same path.
- name: Start local Postgres
working-directory: local
run: docker compose up --wait --wait-timeout 60
run: docker compose up --wait --wait-timeout 60 postgres

# `pnpm test:local` resolves to `vitest run`; the trailing `db-only`
# is a vitest path filter that narrows to __tests__/db-only.test.ts
Expand Down
10 changes: 10 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,16 @@ jobs:
- name: Typecheck (prisma-next — enforces v3 operator-capability gating)
run: pnpm --filter @cipherstash/prisma-next run typecheck

# `packages/bench` is a live importer of `@cipherstash/stack` and
# `@cipherstash/stack-drizzle`, but it has no `test` script (its suites
# need a database), so `pnpm run test` never reaches it and an adapter
# rename could break it unnoticed. Its `build` is `tsc --noEmit`, and
# turbo's `^build` builds the two adapters first. This also runs in the
# Bun job's full `turbo build`, but that job swallows its own test
# failures — the importer guard should not depend on it.
- name: Typecheck (bench — guards the stack/stack-drizzle importers)
run: pnpm exec turbo run build --filter @cipherstash/bench

- name: Lint — no hardcoded package-manager runners
run: pnpm run lint:runners

Expand Down
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ If these variables are missing, tests that require live encryption will fail or
- `packages/wizard`: AI-powered encryption setup (`@cipherstash/wizard`)
- `packages/migrate`: Plaintext-to-encrypted column migration (`@cipherstash/migrate`) — resumable backfill, per-column state
- `packages/prisma-next`: Prisma Next integration (`@cipherstash/prisma-next`) — searchable field-level encryption for Postgres. **EQL v3 only**: per-domain constructors (`cipherstash.TextSearch()` / `text()` / `bigIntOrd()` / …) and `cipherstashFromStack` (the `./v3` and `./stack` entries). The EQL v2 surface was removed — the adapter's baseline migration installs the EQL v3 bundle only (works on Supabase as a non-superuser)
- `packages/stack-drizzle`: Drizzle ORM integration (`@cipherstash/stack-drizzle`), depends on `@cipherstash/stack` — EQL v2 (`.`) and EQL v3 (`./v3`). Split out of `@cipherstash/stack`.
- `packages/stack-drizzle`: Drizzle ORM integration (`@cipherstash/stack-drizzle`), depends on `@cipherstash/stack` — **EQL v3 only**, on the package root (the v2 surface was removed and the old `./v3` subpath collapsed into `.`). Split out of `@cipherstash/stack`.
- `packages/stack-supabase`: Supabase integration (`@cipherstash/stack-supabase`), depends on `@cipherstash/stack` — `encryptedSupabase` (v2) and `encryptedSupabaseV3` (v3). Split out of `@cipherstash/stack`.
- `packages/nextjs`: Next.js helpers and Clerk integration (`./clerk` export)
- `packages/utils`: Shared config (`utils/config`) and logger (`utils/logger`)
Expand Down Expand Up @@ -153,7 +153,7 @@ Three rules to remember when editing CI or pnpm config:
- `encryptQuery(terms[])` for batch query encryption
- **Identity-aware encryption**: Authenticate the client as the end user with `OidcFederationStrategy` (`config.authStrategy`, re-exported from `@cipherstash/stack`), then chain `.withLockContext({ identityClaim })` on operations to bind the data key to a claim. The same claim must be used for encrypt and decrypt. (`LockContext.identify()` from `@cipherstash/stack/identity` is deprecated — the strategy now handles token acquisition; `.withLockContext()` also accepts a `LockContext`.)
- **Integrations**:
- **Drizzle ORM**: `encryptedType`, `extractEncryptionSchema`, `createEncryptionOperators` from `@cipherstash/stack-drizzle` (EQL v3 factories from `@cipherstash/stack-drizzle/v3`)
- **Drizzle ORM**: `types.*` column factories, `extractEncryptionSchema`, `createEncryptionOperators` from `@cipherstash/stack-drizzle`
- **Supabase**: `encryptedSupabase` (v2) / `encryptedSupabaseV3` (v3) from `@cipherstash/stack-supabase`
- **DynamoDB**: `encryptedDynamoDB` from `@cipherstash/stack/dynamodb`

Expand Down
11 changes: 9 additions & 2 deletions packages/bench/__benches__/drizzle/operators.bench.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,15 @@ describe('drizzle', () => {
await handle.db.select().from(benchTable).where(where)
})

bench('like (prefix)', async () => {
const where = (await ops.like(benchTable.encText, '%value-00000%')) as SQL
bench('matches (free-text)', async () => {
// A FULL seeded value, not the shared `value` prefix: every row is
// `value-<7 digits>`, so a bare `value` needle matches all 10k rows and the
// bloom index has nothing to narrow — the number would measure a full scan
// rather than the index path this bench exists to measure.
const where = (await ops.matches(
benchTable.encText,
'value-0000042',
)) as SQL
await handle.db.select().from(benchTable).where(where)
})

Expand Down
51 changes: 25 additions & 26 deletions packages/bench/__tests__/drizzle/operators.explain.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -104,13 +104,19 @@ async function tryExplainWhere(name: string, where: SQL): Promise<void> {

// --- #421: equality + array operators -------------------------------------
//
// `bench_text_hmac_idx` (functional hash on eql_v2.hmac_256) is the expected
// fast path. Pre-fix Drizzle emits bare `=` / `<>` / `IN (...)` which falls
// back to seq scan. Post-fix it emits `eql_v2.hmac_256(col) =
// eql_v2.hmac_256(value)` and the index scan kicks in.
// `bench_text_hmac_idx` (functional hash on eql_v3.eq_term) is the expected
// fast path. The v3 operators do NOT emit that expression directly — they emit
// the wrapper `eql_v3.eq(col, $1::eql_v3.query_text_search)`. It engages the
// index because the wrapper is `LANGUAGE sql IMMUTABLE STRICT` over a single
// SELECT, so the planner inlines it to `eql_v3.eq_term(col) =
// eql_v3.eq_term($1)` — and applies the same inlining to the stored index
// expression, which is how the two meet. Break the inlinability (plpgsql body,
// VOLATILE) and every assertion below silently degrades to a seq scan.
// `scripts/__tests__/bench-index-expressions.test.mjs` pins that contract
// against the shipped bundle without needing a database.
//
// `eq` and `inArray` are naturally high-selectivity (only a few rows match),
// so the planner should pick the hmac index — assertion enforces it.
// so the planner should pick the eq_term index — assertion enforces it.
//
// `ne` and `notInArray` are naturally low-selectivity (almost all rows match);
// even with the hmac index available the planner correctly chooses a seq
Expand Down Expand Up @@ -161,14 +167,12 @@ describe('#421: equality and array operators', () => {
// We don't yet know which call-shaped forms the planner inlines. Record plan
// shape; assertions land in a follow-up once #422 closes.
describe('#422: call-shaped operators (recorded, not asserted)', () => {
it('records like / ilike plan shapes', async () => {
it('records matches plan shape', async () => {
await tryExplainWhere(
'like',
(await ops.like(benchTable.encText, '%value-00000%')) as SQL,
)
await tryExplainWhere(
'ilike',
(await ops.ilike(benchTable.encText, '%VALUE-00000%')) as SQL,
'matches',
// A full seeded value — `value` alone is in every row, so the recorded
// plan would say nothing about the bloom index. See operators.bench.ts.
(await ops.matches(benchTable.encText, 'value-0000042')) as SQL,
)
})

Expand All @@ -190,20 +194,15 @@ describe('#422: call-shaped operators (recorded, not asserted)', () => {
)
})

it('records jsonb operator plan shapes', async () => {
for (const [name, build] of [
[
'jsonbPathQueryFirst',
() => ops.jsonbPathQueryFirst(benchTable.encJsonb, '$.idx'),
],
['jsonbGet', () => ops.jsonbGet(benchTable.encJsonb, '$.idx')],
[
'jsonbPathExists',
() => ops.jsonbPathExists(benchTable.encJsonb, '$.idx'),
],
] as const) {
await tryExplainWhere(name, await build())
}
it('records encrypted-JSONB plan shapes (contains / selector)', async () => {
await tryExplainWhere(
'contains',
(await ops.contains(benchTable.encJsonb, { idx: 42 })) as SQL,
)
await tryExplainWhere(
'selector.gt',
(await ops.selector(benchTable.encJsonb, '$.idx').gt(5000)) as SQL,
)
})

it('records ORDER BY plan shape (asc / desc)', async () => {
Expand Down
2 changes: 2 additions & 0 deletions packages/bench/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
"description": "Performance / index-engagement benchmarks for stack integrations (Drizzle, encryptedSupabase, Prisma).",
"type": "module",
"scripts": {
"build": "tsc --noEmit",
"db:setup": "tsx src/cli/setup.ts",
"db:reset": "tsx src/cli/reset.ts",
"test:local": "vitest run",
Expand All @@ -17,6 +18,7 @@
"pg": "^8.22.0"
},
"devDependencies": {
"@cipherstash/test-kit": "workspace:*",
"@types/node": "^22.20.1",
"@types/pg": "^8.20.0",
"tsx": "catalog:repo",
Expand Down
47 changes: 34 additions & 13 deletions packages/bench/sql/schema.sql
Original file line number Diff line number Diff line change
@@ -1,30 +1,51 @@
-- Bench fixture schema.
-- Single bench table covering text / int / jsonb encrypted columns plus the
-- three canonical EQL functional indexes: hmac_256 (hash), bloom_filter (GIN),
-- ste_vec (GIN).
-- three canonical EQL v3 functional indexes.
--
-- We deliberately do NOT create the `eql_v2.encrypted_operator_class` btree
-- indexes that ore-benches uses. Encrypted composites for full-feature columns
-- (equality + match + ORE) blow past the 2704-byte btree page-size limit, and
-- those indexes don't exist on Supabase anyway — the bench's whole job is to
-- validate that the functional-index path works.
-- Mirrors `src/drizzle/setup.ts`: the columns are concrete `eql_v3_*` Postgres
-- domains (see `types.TextSearch` / `types.IntegerOrd` / `types.Json`). Install
-- the domains and index-term functions first with `stash eql install --eql-version 3`.
--
-- Each index expression must be the expression the matching EQL v3 operator
-- INLINES TO, not merely a term extractor for the same column. The adapter
-- emits a function call (`eql_v3.eq(col, term)`, `eql_v3.matches(...)`,
-- `col @> needle`); every one of those is `LANGUAGE sql IMMUTABLE STRICT`
-- with a single-SELECT body, so the planner inlines it, and Postgres applies
-- the same inlining to the stored index expression. The two only meet if the
-- index is built on the inlined form:
--
-- eql_v3.eq(a, b) -> eql_v3.eq_term(a) = eql_v3.eq_term(b)
-- eql_v3.matches(a, b) -> eql_v3.match_term(a) @> eql_v3.match_term(b)
-- a @> b (json) -> eql_v3.to_ste_vec_query(a)::jsonb
-- @> eql_v3.to_ste_vec_query(b)::jsonb
--
-- That last one is why the JSON index is NOT on `eql_v3.ste_vec(...)`: that
-- function exists and the index builds happily, but nothing the adapter emits
-- ever mentions it, so the index is dead weight. The bundle says so itself, on
-- `eql_v3."@>"(eql_v3_json_search, eql_v3.query_json)`: "Inlines to native
-- `jsonb @>` over `eql_v3.to_ste_vec_query(a)::jsonb`, so a functional GIN
-- index on the same expression engages."
--
-- We deliberately do NOT create btree operator-class indexes: encrypted terms for
-- full-feature columns blow past the 2704-byte btree page-size limit, and the
-- bench's whole job is to validate that the functional-index path works.

DROP TABLE IF EXISTS bench;

CREATE TABLE bench (
id SERIAL PRIMARY KEY,
enc_text eql_v2_encrypted NOT NULL,
enc_int eql_v2_encrypted NOT NULL,
enc_jsonb eql_v2_encrypted NOT NULL
enc_text public.eql_v3_text_search NOT NULL,
enc_int public.eql_v3_integer_ord NOT NULL,
enc_jsonb public.eql_v3_json_search NOT NULL
);

CREATE INDEX bench_text_hmac_idx
ON bench USING hash (eql_v2.hmac_256(enc_text));
ON bench USING hash (eql_v3.eq_term(enc_text));

CREATE INDEX bench_text_bloom_idx
ON bench USING gin (eql_v2.bloom_filter(enc_text));
ON bench USING gin (eql_v3.match_term(enc_text));

CREATE INDEX bench_jsonb_stevec_idx
ON bench USING gin (eql_v2.ste_vec(enc_jsonb));
ON bench USING gin ((eql_v3.to_ste_vec_query(enc_jsonb)::jsonb));

ANALYZE bench;
Loading