diff --git a/.changeset/wasm-inline-model-helpers.md b/.changeset/wasm-inline-model-helpers.md new file mode 100644 index 00000000..40035e71 --- /dev/null +++ b/.changeset/wasm-inline-model-helpers.md @@ -0,0 +1,8 @@ +--- +'@cipherstash/stack': minor +'stash': patch +--- + +`@cipherstash/stack/wasm-inline` now has the model helpers: `encryptModel` / `decryptModel` and `bulkEncryptModels` / `bulkDecryptModels` (#742). They run the same schema traversal as the native entry (shared code, so the two entries cannot drift on which fields get encrypted): declared columns are encrypted — matched by JS property name, nested fields via the column's dotted path — everything else passes through, and `null`/`undefined` fields are preserved without reaching ZeroKMS. A call that encrypts (or decrypts) at least one field is one ZeroKMS round trip regardless of how many fields or models it covers; a `null`/empty batch, or one whose models carry no schema fields, returns without contacting ZeroKMS at all. `types.Date`/`types.Timestamp` columns round-trip `Date` → `Date` (ISO strings on the wire), and failures follow this entry's `{ data } | { failure }` Result contract, with decrypt failures naming every failing field by its model path. Edge code no longer needs the hand-written `bulkEncrypt` field mapping whose failure mode was a schema column silently persisted in plaintext. + +The shared model traversal is also hardened: it no longer mutates the caller's model (previously a nested-column decrypt wrote decrypted plaintext back into the caller's input, and encrypt overwrote it with ciphertext); a literal flat dotted key, a `__proto__`-shaped key, or a non-object model element is handled safely instead of crashing, leaking plaintext, or reaching `Object.prototype`; an already-encrypted field is passed through rather than re-encrypted; and an invalid `Date` is rejected per field. On the WASM entry, model ops now validate the table against the client's schemas, `Date` values are normalized at every encrypt/query crossing (not just the model path), and a `null`/empty model batch returns `{ data: [] }`. The skills update ships in the `stash` tarball, hence the `stash` patch. diff --git a/e2e/wasm/roundtrip.test.ts b/e2e/wasm/roundtrip.test.ts index d7b319a3..df270483 100644 --- a/e2e/wasm/roundtrip.test.ts +++ b/e2e/wasm/roundtrip.test.ts @@ -221,5 +221,72 @@ Deno.test({ ) // Round-trips at the ORIGINAL indices, with the null hole preserved. assertEquals(bulkDecrypted.data, [plaintext, null, second]) + + // 7. (#742) The model helpers — the surface that walks a model against + // its schema, so edge code never hand-rolls the field mapping whose + // failure mode is a column silently persisted in plaintext. Each call + // is one ZeroKMS round trip regardless of field or model count. + const rowA = { id: 'row-a', email: plaintext, note: null } + const modelResult = await client.encryptModel(rowA, users) + assertEquals( + modelResult.failure, + undefined, + `encryptModel() failed: ${modelResult.failure?.message}`, + ) + const encryptedRow = modelResult.data + assertEquals(encryptedRow.id, 'row-a', 'passthrough field was altered') + assertEquals(encryptedRow.note, null, 'null field was altered') + assertEquals( + isEncrypted(encryptedRow.email), + true, + 'schema field was not encrypted by encryptModel', + ) + + const modelBack = await client.decryptModel(encryptedRow, users) + assertEquals( + modelBack.failure, + undefined, + `decryptModel() failed: ${modelBack.failure?.message}`, + ) + assertEquals(modelBack.data, rowA, 'model round-trip mismatch') + + const rowB = { id: 'row-b', email: second, note: 'kept' } + const bulkModels = await client.bulkEncryptModels([rowA, rowB], users) + assertEquals( + bulkModels.failure, + undefined, + `bulkEncryptModels() failed: ${bulkModels.failure?.message}`, + ) + assertEquals(bulkModels.data.length, 2, 'bulkEncryptModels misaligned') + // Assert the payloads are actually ENCRYPTED — otherwise a passthrough + // no-op regression (the plaintext-persistence failure this surface exists + // to prevent) would still round-trip and pass every other assertion. + for (const [i, row] of bulkModels.data.entries()) { + assertEquals( + isEncrypted(row.email), + true, + `bulkEncryptModels[${i}].email is not an encrypted payload`, + ) + assertEquals( + row.note, + i === 0 ? null : 'kept', + 'passthrough field altered', + ) + } + + const bulkModelsBack = await client.bulkDecryptModels( + bulkModels.data, + users, + ) + assertEquals( + bulkModelsBack.failure, + undefined, + `bulkDecryptModels() failed: ${bulkModelsBack.failure?.message}`, + ) + assertEquals( + bulkModelsBack.data, + [rowA, rowB], + 'bulk model round-trip mismatch', + ) }, }) diff --git a/packages/stack/__tests__/wasm-inline-models.test.ts b/packages/stack/__tests__/wasm-inline-models.test.ts new file mode 100644 index 00000000..acb2b9fe --- /dev/null +++ b/packages/stack/__tests__/wasm-inline-models.test.ts @@ -0,0 +1,491 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest' + +// #742: the WASM entry had no model helpers, so edge code hand-rolled the +// field-by-field mapping that `encryptModel` exists to make impossible to get +// wrong (forget one field in the hand-written version and it persists in +// PLAINTEXT). These tests pin the ported surface: schema fields are matched by +// JS property name and addressed at the FFI by DB column name, the whole +// model (or list of models) is ONE FFI crossing, nulls and passthrough fields +// survive verbatim, Date columns round-trip Date → ISO string → Date, and +// failures come back as this entry's `{ failure }` Results with per-field +// coordinates. protect-ffi is mocked; live coverage runs in the Deno e2e. + +const ffi = vi.hoisted(() => ({ + newClient: vi.fn(async () => ({ handle: 'wasm-client' })), + encrypt: vi.fn(async () => ({ v: 3, i: {}, c: 'ct' })), + decrypt: vi.fn(async () => 'plain'), + isEncrypted: vi.fn(() => true), + encryptQuery: vi.fn(async () => ({ v: 3, i: {} })), + encryptQueryBulk: vi.fn( + async (_client: unknown, { queries }: { queries: unknown[] }) => + queries.map((_, n) => ({ v: 3, n })), + ), + encryptBulk: vi.fn( + async (_client: unknown, { plaintexts }: { plaintexts: unknown[] }) => + plaintexts.map((_, n) => ({ v: 3, i: {}, c: `ct-${n}` })), + ), + decryptBulkFallible: vi.fn( + async (_client: unknown, { ciphertexts }: { ciphertexts: unknown[] }) => + ciphertexts.map((_, n) => ({ data: `plain-${n}` })), + ), +})) +vi.mock('@cipherstash/protect-ffi/wasm-inline', () => ffi) +vi.mock('@cipherstash/auth/wasm-inline', () => ({ + AccessKeyStrategy: { + create: vi.fn(() => ({ + data: { getToken: async () => ({ token: 'test' }) }, + })), + }, + OidcFederationStrategy: {}, +})) + +import { encryptedTable, types } from '../src/eql/v3' +import { Encryption } from '../src/wasm-inline' +import { expectData } from './helpers/expect-result' + +// `createdOn` → `created_on` pins the two keyings apart: models are matched +// by JS property name, the FFI payload is addressed by DB column name. +const users = encryptedTable('users', { + email: types.TextEq('email'), + createdOn: types.Date('created_on'), +}) + +// A flat dotted column path — the v3 way to declare a nested field. The +// model carries `{ profile: { ssn } }`; the walk matches it via the path. +const patients = encryptedTable('patients', { + 'profile.ssn': types.TextEq('profile.ssn'), +}) + +async function client() { + return Encryption({ + schemas: [users, patients], + config: { + workspaceCrn: 'crn:test:ws', + accessKey: 'test-key', + clientId: 'id', + clientKey: 'key', + }, + }) +} + +/** A structurally-valid EQL envelope, as `isEncryptedPayload` recognises. */ +const ct = (c: string) => ({ v: 3, i: { t: 'users', c: 'email' }, c }) as never + +describe('WasmEncryptionClient.encryptModel', () => { + beforeEach(() => { + vi.clearAllMocks() + }) + + it('encrypts only schema fields, in one FFI call, addressed by DB name', async () => { + const c = await client() + const out = await c.encryptModel( + { + id: 42, + email: 'alice@example.com', + createdOn: new Date('2026-07-22T00:00:00.000Z'), + role: 'admin', + }, + users, + ) + + expect(ffi.encryptBulk).toHaveBeenCalledTimes(1) + expect(ffi.encrypt).not.toHaveBeenCalled() + const [, opts] = ffi.encryptBulk.mock.calls[0] + expect(opts.plaintexts).toEqual([ + { plaintext: 'alice@example.com', table: 'users', column: 'email' }, + // Date → ISO string: the wasm serde cannot carry a Date, and the DB + // name (`created_on`) proves the property→DB mapping is applied. + { + plaintext: '2026-07-22T00:00:00.000Z', + table: 'users', + column: 'created_on', + }, + ]) + + const data = expectData(out) + // Passthrough fields survive verbatim; schema fields are envelopes. + expect(data.id).toBe(42) + expect(data.role).toBe('admin') + expect(data.email).toEqual({ v: 3, i: {}, c: 'ct-0' }) + expect(data.createdOn).toEqual({ v: 3, i: {}, c: 'ct-1' }) + }) + + it('passes null and undefined schema fields through without encrypting them', async () => { + const c = await client() + // `V3ModelInput` types a schema field as `T | null` — an explicit + // `undefined` is a plain-JS-caller shape, which the runtime preserves + // verbatim rather than encrypting. The cast simulates that caller. + const model = { email: null, createdOn: undefined, id: 1 } + const out = await c.encryptModel( + model as unknown as { email: null; id: number }, + users, + ) + + expect(ffi.encryptBulk).not.toHaveBeenCalled() + // `toStrictEqual`, not `toEqual`: the point of this test is that an + // explicit `undefined` is PRESERVED in place, and `toEqual` treats an + // `undefined` property as absent — it would pass even if `createdOn` were + // dropped entirely. + expect(expectData(out)).toStrictEqual({ + email: null, + createdOn: undefined, + id: 1, + }) + }) + + it('encrypts a nested field declared as a dotted column path', async () => { + const c = await client() + const out = await c.encryptModel( + { profile: { ssn: '123-45-6789', nickname: 'al' } }, + patients, + ) + + const [, opts] = ffi.encryptBulk.mock.calls[0] + expect(opts.plaintexts).toEqual([ + { plaintext: '123-45-6789', table: 'patients', column: 'profile.ssn' }, + ]) + const data = expectData(out) as { + profile: { ssn: unknown; nickname: string } + } + expect(data.profile.ssn).toEqual({ v: 3, i: {}, c: 'ct-0' }) + expect(data.profile.nickname).toBe('al') + }) + + it('rejects an out-of-range numeric plaintext as a { failure }, before the FFI', async () => { + const c = await client() + // The type system already rejects this shape — the cast simulates a + // plain-JS caller, which is exactly who the runtime guard exists for. + const out = await c.encryptModel( + { email: Number.NaN } as unknown as { email: string }, + users, + ) + + expect(out.failure?.type).toBe('EncryptionError') + expect(ffi.encryptBulk).not.toHaveBeenCalled() + }) + + it('refuses a short FFI response instead of silently dropping fields', async () => { + ffi.encryptBulk.mockResolvedValueOnce([{ v: 3, i: {}, c: 'only-one' }]) + + const c = await client() + const out = await c.encryptModel( + { email: 'a@b.com', createdOn: new Date(0) }, + users, + ) + + expect(out.failure?.type).toBe('EncryptionError') + expect(out.failure?.message).toMatch(/sent 2 payload\(s\).*received 1/) + }) +}) + +describe('WasmEncryptionClient.decryptModel', () => { + beforeEach(() => { + vi.clearAllMocks() + }) + + it('decrypts every envelope in one FFI call and leaves the rest alone', async () => { + const c = await client() + const out = await c.decryptModel( + { id: 7, email: ct('a'), note: 'plain', missing: null }, + users, + ) + + expect(ffi.decryptBulkFallible).toHaveBeenCalledTimes(1) + const [, opts] = ffi.decryptBulkFallible.mock.calls[0] + expect(opts.ciphertexts).toEqual([{ ciphertext: ct('a') }]) + + expect(expectData(out)).toEqual({ + id: 7, + email: 'plain-0', + note: 'plain', + missing: null, + }) + }) + + it('rebuilds date-like columns into Date values from cast_as', async () => { + ffi.decryptBulkFallible.mockResolvedValueOnce([ + { data: '2026-07-22T01:02:03.000Z' }, + ] as never) + + const c = await client() + const out = await c.decryptModel({ createdOn: ct('d') }, users) + + const data = expectData(out) + expect(data.createdOn).toBeInstanceOf(Date) + expect((data.createdOn as Date).toISOString()).toBe( + '2026-07-22T01:02:03.000Z', + ) + }) + + it('names every failed field by its model path', async () => { + ffi.decryptBulkFallible.mockResolvedValueOnce([ + { error: 'boom', code: 'CT_ERROR' }, + ] as never) + + const c = await client() + const out = await c.decryptModel({ profile: { ssn: ct('x') } }, patients) + + expect(out.failure?.type).toBe('DecryptionError') + expect(out.failure?.message).toMatch(/failed for 1 of 1 payload\(s\)/) + expect(out.failure?.message).toContain('profile.ssn (CT_ERROR): boom') + }) +}) + +describe('WasmEncryptionClient.bulkEncryptModels', () => { + beforeEach(() => { + vi.clearAllMocks() + }) + + it('batches every field of every model into one FFI call', async () => { + const c = await client() + const out = await c.bulkEncryptModels( + [ + { email: 'a@b.com', id: 1 }, + { email: 'c@d.com', id: 2 }, + ], + users, + ) + + expect(ffi.encryptBulk).toHaveBeenCalledTimes(1) + const [, opts] = ffi.encryptBulk.mock.calls[0] + expect(opts.plaintexts).toHaveLength(2) + + const data = expectData(out) + expect(data).toEqual([ + { email: { v: 3, i: {}, c: 'ct-0' }, id: 1 }, + { email: { v: 3, i: {}, c: 'ct-1' }, id: 2 }, + ]) + }) + + it('returns an empty array for an empty batch, with no FFI call', async () => { + const c = await client() + expect(await c.bulkEncryptModels([], users)).toEqual({ data: [] }) + expect(ffi.encryptBulk).not.toHaveBeenCalled() + }) +}) + +describe('WasmEncryptionClient.bulkDecryptModels', () => { + beforeEach(() => { + vi.clearAllMocks() + }) + + it('maps decrypted fields back to the right models from one FFI call', async () => { + const c = await client() + const out = await c.bulkDecryptModels( + [ + { email: ct('a'), id: 1 }, + { email: ct('b'), id: 2, note: null }, + ], + users, + ) + + expect(ffi.decryptBulkFallible).toHaveBeenCalledTimes(1) + expect(expectData(out)).toEqual([ + { email: 'plain-0', id: 1 }, + { email: 'plain-1', id: 2, note: null }, + ]) + }) + + it('labels failures with the model index and field path', async () => { + ffi.decryptBulkFallible.mockResolvedValueOnce([ + { data: 'ok' }, + { error: 'boom' }, + ] as never) + + const c = await client() + const out = await c.bulkDecryptModels( + [{ email: ct('a') }, { email: ct('b') }], + users, + ) + + expect(out.failure?.type).toBe('DecryptionError') + expect(out.failure?.message).toContain('[model 1] email: boom') + }) + + it('returns an empty array for an empty batch, with no FFI call', async () => { + const c = await client() + expect(await c.bulkDecryptModels([], users)).toEqual({ data: [] }) + expect(ffi.decryptBulkFallible).not.toHaveBeenCalled() + }) +}) + +// #742 review: the shared traversal was rewritten to be non-mutating and +// unambiguous. These pin the behaviours that were bugs before the rewrite. +describe('WasmEncryptionClient model helpers — hardening (#742 review)', () => { + beforeEach(() => { + vi.clearAllMocks() + }) + + it('does not mutate the caller model on encrypt (nested column)', async () => { + const c = await client() + const input = { profile: { ssn: '123-45-6789', nick: 'al' } } + const out = await c.encryptModel(input, patients) + + // The caller's own object is untouched — no envelope written back in. + expect(input.profile.ssn).toBe('123-45-6789') + expect(input.profile.nick).toBe('al') + const data = expectData(out) as { profile: { ssn: unknown; nick: string } } + expect(data.profile.ssn).toEqual({ v: 3, i: {}, c: 'ct-0' }) + // The returned object is independent of the input. + expect(data.profile).not.toBe(input.profile) + }) + + it('does not write decrypted plaintext back into the caller (nested)', async () => { + const c = await client() + const envelope = ct('a') + const input = { profile: { ssn: envelope } } + const out = await c.decryptModel(input, patients) + + // The input the caller believes is still encrypted is unchanged. + expect(input.profile.ssn).toBe(envelope) + const data = expectData(out) as { profile: { ssn: unknown } } + expect(data.profile.ssn).toBe('plain-0') + }) + + it('normalizes a literal flat dotted key without crashing or leaking plaintext', async () => { + const c = await client() + const out = await c.encryptModel( + { 'profile.ssn': 'secret' } as never, + patients, + ) + const data = expectData(out) as Record & { + profile?: { ssn?: unknown } + } + // The flat plaintext key is gone; the field is encrypted at its column path. + expect(data['profile.ssn']).toBeUndefined() + expect(data.profile?.ssn).toEqual({ v: 3, i: {}, c: 'ct-0' }) + }) + + it('passes an already-encrypted schema field through without re-encrypting', async () => { + const c = await client() + const existing = ct('already') + const out = await c.encryptModel({ email: existing, id: 1 } as never, users) + + expect(ffi.encryptBulk).not.toHaveBeenCalled() + const data = expectData(out) as { email: unknown; id: number } + expect(data.email).toBe(existing) + expect(data.id).toBe(1) + }) + + it('rejects a non-object model element with a clear failure', async () => { + const c = await client() + const out = await c.bulkEncryptModels(['x@y.com'] as never, users) + expect(out.failure?.type).toBe('EncryptionError') + expect(out.failure?.message).toContain( + 'each model must be a non-null object', + ) + expect(ffi.encryptBulk).not.toHaveBeenCalled() + }) + + it('returns { data: [] } for a null or undefined models argument', async () => { + const c = await client() + expect(await c.bulkEncryptModels(null as never, users)).toEqual({ + data: [], + }) + expect(await c.bulkDecryptModels(undefined as never, users)).toEqual({ + data: [], + }) + expect(ffi.encryptBulk).not.toHaveBeenCalled() + expect(ffi.decryptBulkFallible).not.toHaveBeenCalled() + }) + + it('fails decrypt against a table the client was not initialized with', async () => { + const c = await client() + const foreign = encryptedTable('foreign', { x: types.TextEq('x') }) + const out = await c.decryptModel({ x: ct('a') }, foreign) + expect(out.failure?.type).toBe('DecryptionError') + expect(out.failure?.message).toContain( + 'not one this client was initialized with', + ) + expect(ffi.decryptBulkFallible).not.toHaveBeenCalled() + }) + + it('fails encrypt against a table the client was not initialized with', async () => { + const c = await client() + const foreign = encryptedTable('foreign', { x: types.TextEq('x') }) + // Both encrypt entrypoints call `requireTable`; cover them symmetrically + // with the decrypt case above. + const single = await c.encryptModel({ x: 'secret' }, foreign) + expect(single.failure?.type).toBe('EncryptionError') + expect(single.failure?.message).toContain( + 'not one this client was initialized with', + ) + const bulk = await c.bulkEncryptModels([{ x: 'secret' }], foreign) + expect(bulk.failure?.type).toBe('EncryptionError') + expect(bulk.failure?.message).toContain( + 'not one this client was initialized with', + ) + expect(ffi.encryptBulk).not.toHaveBeenCalled() + }) + + it('preserves a passthrough field literally named __proto__', async () => { + const c = await client() + // A non-schema, non-envelope field named `__proto__` (as `JSON.parse` + // materialises it) is a passthrough value. Plain `out.__proto__ = value` + // would hit the prototype setter and silently drop it; it must survive + // verbatim as an own property, and the global prototype must stay intact. + const model = JSON.parse('{"id":1,"__proto__":{"kept":true}}') + const out = await c.encryptModel(model, users) + + expect(ffi.encryptBulk).not.toHaveBeenCalled() + const data = expectData(out) as Record + // Stored as an OWN data property, not swallowed by the prototype setter. + expect(Object.getOwnPropertyNames(data)).toContain('__proto__') + expect(Object.getOwnPropertyDescriptor(data, '__proto__')?.value).toEqual({ + kept: true, + }) + // Untouched global prototype: no `kept` leaked onto `Object.prototype`. + expect('kept' in {}).toBe(false) + }) + + it('rejects an invalid Date with a field-named failure, before the FFI', async () => { + const c = await client() + const out = await c.encryptModel( + { createdOn: new Date(Number.NaN) } as never, + users, + ) + expect(out.failure?.type).toBe('EncryptionError') + expect(out.failure?.message).toContain('createdOn') + expect(ffi.encryptBulk).not.toHaveBeenCalled() + }) + + it('refuses a __proto__ dotted key instead of polluting the prototype', async () => { + const c = await client() + // A DB-influenced model with a prototype-shaped own key holding an envelope. + const malicious = JSON.parse( + `{"__proto__.toString": ${JSON.stringify(ct('x'))}}`, + ) + const out = await c.decryptModel(malicious, patients) + + expect(out.failure?.type).toBe('DecryptionError') + expect(out.failure?.message).toContain('Forbidden key') + // The global prototype is intact. + expect(typeof {}.toString).toBe('function') + }) + + it('rebuilds a valid Date but keeps an unparseable date value as-is', async () => { + ffi.decryptBulkFallible.mockResolvedValueOnce([ + { data: 'not-a-date' }, + ] as never) + const c = await client() + const out = await c.decryptModel({ createdOn: ct('d') }, users) + // No silent Invalid Date object — the raw value is returned instead. + expect(expectData(out).createdOn).toBe('not-a-date') + }) + + it('normalizes a Date at the value-level bulkEncrypt crossing (no `{}` corruption)', async () => { + const c = await client() + await c.bulkEncrypt([ + { + // A plain-JS caller can pass a Date the WasmPlaintext type forbids. + plaintext: new Date('2026-07-22T00:00:00.000Z') as never, + table: users, + column: users.createdOn, + }, + ]) + const opts = ffi.encryptBulk.mock.calls[0][1] as { + plaintexts: Array<{ plaintext: unknown }> + } + expect(opts.plaintexts[0].plaintext).toBe('2026-07-22T00:00:00.000Z') + }) +}) diff --git a/packages/stack/src/encryption/helpers/model-helpers.ts b/packages/stack/src/encryption/helpers/model-helpers.ts index 1d7eec4f..b321ecd4 100644 --- a/packages/stack/src/encryption/helpers/model-helpers.ts +++ b/packages/stack/src/encryption/helpers/model-helpers.ts @@ -4,42 +4,24 @@ import { encryptBulk, } from '@cipherstash/protect-ffi' import { isEncryptedPayload } from '@/encryption/helpers' -import { assertValidNumericValue } from '@/encryption/helpers/validation' +import { + fieldsForModelIndex, + prepareBulkModelsForOperation, + prepareFieldsForDecryption, + prepareFieldsForEncryption, + resolveEncryptColumnMap, + setNestedValue, +} from '@/encryption/helpers/model-traversal' import type { AuditData } from '@/encryption/operations/base-operation' import type { Context } from '@/identity' import type { BuildableTable, Client, Decrypted, Encrypted } from '@/types' -/** - * Sets a value at a nested path in an object, creating intermediate objects as needed. - * Includes prototype pollution protection. - */ -function setNestedValue( - obj: Record, - path: string[], - value: unknown, -): void { - const FORBIDDEN_KEYS = ['__proto__', 'prototype', 'constructor'] - let current: Record = obj - for (let i = 0; i < path.length - 1; i++) { - const part = path[i] - if (FORBIDDEN_KEYS.includes(part)) { - throw new Error(`[encryption]: Forbidden key "${part}" in field path`) - } - if ( - !(part in current) || - typeof current[part] !== 'object' || - current[part] === null - ) { - current[part] = {} - } - current = current[part] as Record - } - const lastKey = path[path.length - 1] - if (FORBIDDEN_KEYS.includes(lastKey)) { - throw new Error(`[encryption]: Forbidden key "${lastKey}" in field path`) - } - current[lastKey] = value -} +// The traversal layer (field matching, null bookkeeping, nested placement) +// lives in `./model-traversal` so the WASM entry shares it verbatim — see that +// module's header. This module pairs it with the NATIVE protect-ffi batch +// calls and the lock-context variants. `resolveEncryptColumnMap` is re-exported +// because it is part of this module's documented surface. +export { resolveEncryptColumnMap } from '@/encryption/helpers/model-traversal' /** * Helper function to extract encrypted fields from a model @@ -151,157 +133,6 @@ async function handleMultiModelBulkOperation( return mappedResults } -/** - * Helper function to prepare fields for decryption - */ -function prepareFieldsForDecryption>( - model: T, -): { - otherFields: Record - operationFields: Record - keyMap: Record - nullFields: Record -} { - const otherFields = { ...model } as Record - const operationFields: Record = {} - const nullFields: Record = {} - const keyMap: Record = {} - let index = 0 - - const processNestedFields = (obj: Record, prefix = '') => { - for (const [key, value] of Object.entries(obj)) { - const fullKey = prefix ? `${prefix}.${key}` : key - - if (value === null || value === undefined) { - nullFields[fullKey] = value - continue - } - - if (typeof value === 'object' && !isEncryptedPayload(value)) { - // Recursively process nested objects - processNestedFields(value as Record, fullKey) - } else if (isEncryptedPayload(value)) { - // This is an encrypted field - const id = index.toString() - keyMap[id] = fullKey - operationFields[fullKey] = value - index++ - - // Remove from otherFields - const parts = fullKey.split('.') - let current = otherFields - for (let i = 0; i < parts.length - 1; i++) { - current = current[parts[i]] as Record - } - delete current[parts[parts.length - 1]] - } - } - } - - processNestedFields(model) - return { otherFields, operationFields, keyMap, nullFields } -} - -/** - * Helper function to prepare fields for encryption - */ -/** - * Resolve how a table's model fields map onto encrypt-config columns. - * - * `columnPaths` are the keys used to MATCH a user model's fields (the JS - * property names); `toColumnName` maps a matched field to the name the FFI / - * encrypt config is keyed by (the DB name). - * - * When a table exposes `buildColumnKeyMap()` (v3), those two can differ, so we - * match by property but address by DB name. Otherwise (v2) `build()` already - * keys columns by the property name, so both are that same key (identity map). - */ -export function resolveEncryptColumnMap(table: BuildableTable): { - columnPaths: string[] - toColumnName: (path: string) => string -} { - const keyMap = table.buildColumnKeyMap?.() - if (keyMap) { - return { - columnPaths: Object.keys(keyMap), - toColumnName: (path) => keyMap[path] ?? path, - } - } - const columnPaths = Object.keys(table.build().columns) - return { columnPaths, toColumnName: (path) => path } -} - -function prepareFieldsForEncryption>( - model: T, - table: BuildableTable, -): { - otherFields: Record - operationFields: Record - keyMap: Record - nullFields: Record -} { - const otherFields = { ...model } as Record - const operationFields: Record = {} - const nullFields: Record = {} - const keyMap: Record = {} - let index = 0 - - const processNestedFields = ( - obj: Record, - prefix = '', - columnPaths: string[] = [], - ) => { - for (const [key, value] of Object.entries(obj)) { - const fullKey = prefix ? `${prefix}.${key}` : key - - if (value === null || value === undefined) { - nullFields[fullKey] = value - continue - } - - if ( - typeof value === 'object' && - !isEncryptedPayload(value) && - !columnPaths.includes(fullKey) - ) { - // Only process nested objects if they're in the schema - if (columnPaths.some((path) => path.startsWith(fullKey))) { - processNestedFields( - value as Record, - fullKey, - columnPaths, - ) - } - } else if (columnPaths.includes(fullKey)) { - // Only process fields that are explicitly defined in the schema. - // Reject an out-of-range numeric plaintext (NaN/±Infinity for `number`, - // outside i64 for `bigint`) here — the single-value/query paths guard - // at their own boundary, but the model path builds the FFI payload - // directly, so validate per field before it reaches protect-ffi. - assertValidNumericValue(value) - const id = index.toString() - keyMap[id] = fullKey - operationFields[fullKey] = value - index++ - - // Remove from otherFields - const parts = fullKey.split('.') - let current = otherFields - for (let i = 0; i < parts.length - 1; i++) { - current = current[parts[i]] as Record - } - delete current[parts[parts.length - 1]] - } - } - } - - // Get all column paths from the table schema (matched by JS property name). - const { columnPaths } = resolveEncryptColumnMap(table) - processNestedFields(model, '', columnPaths) - - return { otherFields, operationFields, keyMap, nullFields } -} - /** * Helper function to convert a model with encrypted fields to a decrypted model */ @@ -524,159 +355,6 @@ export async function encryptModelFieldsWithLockContext( return result } -/** - * Helper function to prepare multiple models for bulk operation - */ -function prepareBulkModelsForOperation>( - models: T[], - table?: BuildableTable, -): { - otherFields: Record[] - operationFields: Record[] - keyMap: Record - nullFields: Record[] -} { - const otherFields: Record[] = [] - const operationFields: Record[] = [] - const nullFields: Record[] = [] - const keyMap: Record = {} - let index = 0 - - for (let modelIndex = 0; modelIndex < models.length; modelIndex++) { - const model = models[modelIndex] - const modelOtherFields = { ...model } as Record - const modelOperationFields: Record = {} - const modelNullFields: Record = {} - - const processNestedFields = ( - obj: Record, - prefix = '', - columnPaths: string[] = [], - ) => { - for (const [key, value] of Object.entries(obj)) { - const fullKey = prefix ? `${prefix}.${key}` : key - - if (value === null || value === undefined) { - modelNullFields[fullKey] = value - continue - } - - if ( - typeof value === 'object' && - !isEncryptedPayload(value) && - !columnPaths.includes(fullKey) - ) { - // Only process nested objects if they're in the schema - if (columnPaths.some((path) => path.startsWith(fullKey))) { - processNestedFields( - value as Record, - fullKey, - columnPaths, - ) - } - } else if (columnPaths.includes(fullKey)) { - // Only process fields that are explicitly defined in the schema. - // Reject an out-of-range numeric plaintext (NaN/±Infinity for - // `number`, outside i64 for `bigint`) before it reaches the bulk FFI - // payload — the bulk path builds that payload directly. This arm runs - // only for encryption (`if (table)`); the decrypt walker below does - // not validate. - assertValidNumericValue(value) - const id = index.toString() - keyMap[id] = { modelIndex, fieldKey: fullKey } - modelOperationFields[fullKey] = value - index++ - - // Remove from otherFields - const parts = fullKey.split('.') - let current = modelOtherFields - for (let i = 0; i < parts.length - 1; i++) { - current = current[parts[i]] as Record - } - delete current[parts[parts.length - 1]] - } - } - } - - if (table) { - // Get all column paths from the table schema (matched by JS property name). - const { columnPaths } = resolveEncryptColumnMap(table) - processNestedFields(model, '', columnPaths) - } else { - // For decryption, process all encrypted fields - const processEncryptedFields = ( - obj: Record, - prefix = '', - columnPaths: string[] = [], - ) => { - for (const [key, value] of Object.entries(obj)) { - const fullKey = prefix ? `${prefix}.${key}` : key - - if (value === null || value === undefined) { - modelNullFields[fullKey] = value - continue - } - - if ( - typeof value === 'object' && - !isEncryptedPayload(value) && - !columnPaths.includes(fullKey) - ) { - // Recursively process nested objects - processEncryptedFields( - value as Record, - fullKey, - columnPaths, - ) - } else if (isEncryptedPayload(value)) { - // This is an encrypted field - const id = index.toString() - keyMap[id] = { modelIndex, fieldKey: fullKey } - modelOperationFields[fullKey] = value - index++ - - // Remove from otherFields - const parts = fullKey.split('.') - let current = modelOtherFields - for (let i = 0; i < parts.length - 1; i++) { - current = current[parts[i]] as Record - } - delete current[parts[parts.length - 1]] - } - } - } - processEncryptedFields(model) - } - - otherFields.push(modelOtherFields) - operationFields.push(modelOperationFields) - nullFields.push(modelNullFields) - } - - return { otherFields, operationFields, keyMap, nullFields } -} - -/** - * Collect the per-model fields out of a bulk-operation result map keyed by - * `${modelIndex}-${fieldKey}` ids, splitting each id at the FIRST hyphen - * only. Field keys may themselves contain hyphens (a `some-field` column, or - * a nested `profile.some-field` path), so a naive `split('-')` would truncate - * the field key at its first hyphen and silently drop the value during model - * reconstruction. - */ -function fieldsForModelIndex( - fields: Record, - modelIndex: number, -): Record { - const result: Record = {} - for (const [id, value] of Object.entries(fields)) { - const sep = id.indexOf('-') - if (Number.parseInt(id.slice(0, sep), 10) !== modelIndex) continue - result[id.slice(sep + 1)] = value - } - return result -} - /** * Helper function to convert multiple decrypted models to models with encrypted fields */ diff --git a/packages/stack/src/encryption/helpers/model-traversal.ts b/packages/stack/src/encryption/helpers/model-traversal.ts new file mode 100644 index 00000000..f00d4fa0 --- /dev/null +++ b/packages/stack/src/encryption/helpers/model-traversal.ts @@ -0,0 +1,404 @@ +import { isEncryptedPayload } from '@/encryption/helpers' +import { assertValidNumericValue } from '@/encryption/helpers/validation' +import type { BuildableTable } from '@/types' + +/** + * The pure model-traversal layer shared by BOTH encryption entries: the + * native model helpers (`./model-helpers`, which pair it with the NAPI + * `encryptBulk`/`decryptBulk`) and the WASM entry (`@/wasm-inline`, which + * pairs it with the `/wasm-inline` FFI batch calls — #742). + * + * Extracted so the walk itself cannot drift between entries. The traversal is + * where the quiet plaintext-leak class of bug lives — a schema field the walk + * fails to match is passed through UNENCRYPTED — so there must be exactly one + * implementation of it. + * + * MUST stay free of runtime `@cipherstash/protect-ffi` imports (type-only is + * fine): this module is bundled into `dist/wasm-inline.js`, whose whole point + * is to not touch the native binding. `__tests__/wasm-inline-bundle-isolation` + * enforces that at the artifact level. + */ + +/** + * Sets a value at a nested path in an object, creating intermediate objects as needed. + * Includes prototype pollution protection. + * + * Intermediates are always plain objects, never arrays, so a hypothetical + * array-indexed column path (`items.0.secret`) rebuilds under `{ '0': … }` + * rather than `[ … ]`. Schema columns are declared by name, not index, so this + * has no known caller; passthrough arrays keep their shape via the array arm of + * `buildPassthroughTree`. + */ +export function setNestedValue( + obj: Record, + path: string[], + value: unknown, +): void { + const FORBIDDEN_KEYS = ['__proto__', 'prototype', 'constructor'] + let current: Record = obj + for (let i = 0; i < path.length - 1; i++) { + const part = path[i] + if (FORBIDDEN_KEYS.includes(part)) { + throw new Error(`[encryption]: Forbidden key "${part}" in field path`) + } + if ( + !(part in current) || + typeof current[part] !== 'object' || + current[part] === null + ) { + current[part] = {} + } + current = current[part] as Record + } + const lastKey = path[path.length - 1] + if (FORBIDDEN_KEYS.includes(lastKey)) { + throw new Error(`[encryption]: Forbidden key "${lastKey}" in field path`) + } + current[lastKey] = value +} + +/** + * Resolve how a table's model fields map onto encrypt-config columns. + * + * `columnPaths` are the keys used to MATCH a user model's fields (the JS + * property names); `toColumnName` maps a matched field to the name the FFI / + * encrypt config is keyed by (the DB name). + * + * When a table exposes `buildColumnKeyMap()` (v3), those two can differ, so we + * match by property but address by DB name. Otherwise (v2) `build()` already + * keys columns by the property name, so both are that same key (identity map). + */ +export function resolveEncryptColumnMap(table: BuildableTable): { + columnPaths: string[] + toColumnName: (path: string) => string +} { + const keyMap = table.buildColumnKeyMap?.() + if (keyMap) { + return { + columnPaths: Object.keys(keyMap), + toColumnName: (path) => keyMap[path] ?? path, + } + } + const columnPaths = Object.keys(table.build().columns) + return { columnPaths, toColumnName: (path) => path } +} + +// --------------------------------------------------------------------------- +// The non-mutating walk (#742 review) +// --------------------------------------------------------------------------- +// +// The previous walk shallow-copied the model (`{ ...model }`) and then +// `delete`d each matched field out of the copy. Because a shallow copy shares +// every nested object with the caller, that `delete` mutated the CALLER's +// input — and on a decrypt the sibling rebuild wrote decrypted PLAINTEXT back +// into the object the caller believed was still encrypted. Locating the field +// to delete meant re-`split('.')`-ing the dotted key and walking from the copy, +// which crashed (or leaked plaintext) on a literal flat dotted key and could +// reach `Object.prototype` via a `__proto__.x` key. +// +// This builder never touches the input. It returns a FRESH tree that omits the +// operation fields and keeps nulls / passthrough values in place, so: +// - the caller's model is never mutated (no shared nested object is written); +// - a literal flat dotted key is simply omitted, not re-split — no crash, no +// surviving plaintext, no `Object.prototype` reach; +// - nulls stay where they sit, so a dotted null key no longer materialises a +// phantom nested object on rebuild (the rebuild no longer re-applies a +// separate null map). + +/** + * True only for a plain object (`{}` / `Object.create(null)`) or an array — + * the containers the walk descends into and clones. A `Date`, or any other + * class instance, is NOT a container: cloning it by iterating its enumerable + * keys would rebuild it as an empty `{}` (a `Date` has none), destroying the + * value. Those pass through by reference instead (they are never mutated). + */ +function isPlainContainer(value: object): boolean { + if (Array.isArray(value)) return true + const proto = Object.getPrototypeOf(value) + return proto === Object.prototype || proto === null +} + +interface WalkHooks { + /** True when this field should be handed to the FFI (a schema column on + * encrypt, an EQL payload on decrypt) and omitted from the passthrough + * tree. */ + isOperationField: (fullKey: string, value: unknown) => boolean + /** True when the walk should descend into this object because a schema + * column lives under it (always true on the schema-blind decrypt walk). */ + shouldRecurse: (fullKey: string) => boolean + /** Receives each operation field in visit order. */ + onOperationField: (fullKey: string, value: unknown) => void +} + +/** + * Reject a top-level model that is not a plain record. A shallow-spread of a + * string (`{ ...'x@y' }`) explodes into a char-indexed object and a + * number/boolean into `{}`, which the old walk returned as a successful (but + * silently wrong) result; fail loudly instead (#742 review). + */ +function assertModelObject( + model: unknown, +): asserts model is Record { + if (typeof model !== 'object' || model === null || Array.isArray(model)) { + throw new Error('[encryption]: each model must be a non-null object') + } +} + +/** + * Pre-FFI guards for a value about to be encrypted from the model path (the + * single/query paths guard at their own boundary). Out-of-range numbers and an + * invalid `Date` (`new Date(NaN)`) are rejected here, per field, so the failure + * names the column rather than surfacing as one coordinate-less batch error + * from inside the FFI. + */ +function assertEncryptableValue(value: unknown, fullKey: string): void { + assertValidNumericValue(value) + if (value instanceof Date && Number.isNaN(value.getTime())) { + throw new Error( + `[encryption]: field "${fullKey}" is an invalid Date and cannot be encrypted`, + ) + } +} + +/** + * Store a passthrough field as an OWN property. Plain `out[key] = value` would, + * for a field literally named `__proto__` (e.g. straight off `JSON.parse`), + * invoke the prototype setter instead of storing the value — silently dropping + * the field (a fidelity gap, not a security one: it only mutates this fresh + * `out`, never global `Object.prototype`). `defineProperty` keeps the shape + * verbatim, mirroring the `setNestedValue` guard on the operation-field path. + */ +function assignOwn( + out: Record, + key: string, + value: unknown, +): void { + if (key === '__proto__') { + Object.defineProperty(out, key, { + value, + writable: true, + enumerable: true, + configurable: true, + }) + } else { + out[key] = value + } +} + +/** + * Walk `obj`, returning a fresh passthrough tree that OMITS every operation + * field and streams those fields to `hooks.onOperationField` in visit order. + * Arrays are preserved as arrays. Never mutates `obj`. + */ +function buildPassthroughTree( + obj: Record, + prefix: string, + hooks: WalkHooks, +): unknown { + const out: Record = {} + + for (const [key, value] of Object.entries(obj)) { + const fullKey = prefix ? `${prefix}.${key}` : key + + // Nulls carry no schema meaning and are never an operation — keep them in + // place so the output shape matches the input exactly. + if (value === null || value === undefined) { + assignOwn(out, key, value) + continue + } + + if (hooks.isOperationField(fullKey, value)) { + hooks.onOperationField(fullKey, value) + // Omitted from `out`; the caller places the FFI result back here. + continue + } + + if ( + typeof value === 'object' && + isPlainContainer(value) && + !isEncryptedPayload(value) && + hooks.shouldRecurse(fullKey) + ) { + assignOwn( + out, + key, + buildPassthroughTree(value as Record, fullKey, hooks), + ) + } else { + // Passthrough: a scalar, a Date / class instance, an already-encrypted + // payload the schema didn't claim, or a plain object with no schema + // column beneath it. Not mutated, so sharing the reference is safe. + assignOwn(out, key, value) + } + } + + // Preserve array shape. An operation field inside an array is omitted, + // leaving a hole at that index; the caller sets the FFI result back by path. + if (Array.isArray(obj)) { + const arr: unknown[] = [] + for (const [key, value] of Object.entries(out)) { + arr[Number(key)] = value + } + return arr + } + return out +} + +/** Recurse into an object only when a schema column lives at or under it. */ +function encryptShouldRecurse( + columnPaths: readonly string[], +): (fullKey: string) => boolean { + return (fullKey) => + columnPaths.some( + (path) => path === fullKey || path.startsWith(`${fullKey}.`), + ) +} + +/** + * Helper function to prepare fields for decryption + */ +export function prepareFieldsForDecryption>( + model: T, +): { + otherFields: Record + operationFields: Record + keyMap: Record + nullFields: Record +} { + assertModelObject(model) + const operationFields: Record = {} + const keyMap: Record = {} + let index = 0 + + // Top-level model is a non-array object (asserted above), so the tree is a + // Record; `buildPassthroughTree` returns `unknown` only to type its array arm. + const otherFields = buildPassthroughTree(model, '', { + isOperationField: (_fullKey, value) => isEncryptedPayload(value), + shouldRecurse: () => true, + onOperationField: (fullKey, value) => { + keyMap[index.toString()] = fullKey + operationFields[fullKey] = value + index++ + }, + }) as Record + + // Nulls are retained in `otherFields` in place, so no separate null map is + // re-applied on rebuild (which is what materialised phantom nested objects). + return { otherFields, operationFields, keyMap, nullFields: {} } +} + +/** + * Helper function to prepare fields for encryption + */ +export function prepareFieldsForEncryption>( + model: T, + table: BuildableTable, +): { + otherFields: Record + operationFields: Record + keyMap: Record + nullFields: Record +} { + assertModelObject(model) + const { columnPaths } = resolveEncryptColumnMap(table) + const columnSet = new Set(columnPaths) + const operationFields: Record = {} + const keyMap: Record = {} + let index = 0 + + const otherFields = buildPassthroughTree(model, '', { + // Skip a value that is ALREADY an EQL payload even at a schema path — it + // has been encrypted before (a read-modify-write of a fetched row); the + // old walk re-encrypted it, silently for a `types.Json` column (#742). + isOperationField: (fullKey, value) => + columnSet.has(fullKey) && !isEncryptedPayload(value), + shouldRecurse: encryptShouldRecurse(columnPaths), + onOperationField: (fullKey, value) => { + assertEncryptableValue(value, fullKey) + keyMap[index.toString()] = fullKey + operationFields[fullKey] = value + index++ + }, + }) as Record + + return { otherFields, operationFields, keyMap, nullFields: {} } +} + +/** + * Helper function to prepare multiple models for bulk operation + */ +export function prepareBulkModelsForOperation< + T extends Record, +>( + models: T[], + table?: BuildableTable, +): { + otherFields: Record[] + operationFields: Record[] + keyMap: Record + nullFields: Record[] +} { + const otherFields: Record[] = [] + const operationFields: Record[] = [] + const keyMap: Record = {} + let index = 0 + + // Column paths are row-invariant, so resolve the map once for the whole batch + // rather than once per model (the old walk rebuilt it inside the loop). + const columnPaths = table ? resolveEncryptColumnMap(table).columnPaths : [] + const columnSet = new Set(columnPaths) + const shouldRecurse = table ? encryptShouldRecurse(columnPaths) : () => true + const isOperationField = table + ? (fullKey: string, value: unknown) => + columnSet.has(fullKey) && !isEncryptedPayload(value) + : (_fullKey: string, value: unknown) => isEncryptedPayload(value) + + for (let modelIndex = 0; modelIndex < models.length; modelIndex++) { + const model = models[modelIndex] + assertModelObject(model) + const modelOperationFields: Record = {} + + const tree = buildPassthroughTree(model, '', { + isOperationField, + shouldRecurse, + onOperationField: (fullKey, value) => { + // Only the encrypt walk validates; the decrypt walk collects payloads. + if (table) assertEncryptableValue(value, fullKey) + keyMap[index.toString()] = { modelIndex, fieldKey: fullKey } + modelOperationFields[fullKey] = value + index++ + }, + }) as Record + + otherFields.push(tree) + operationFields.push(modelOperationFields) + } + + return { + otherFields, + operationFields, + keyMap, + nullFields: models.map(() => ({})), + } +} + +/** + * Collect the per-model fields out of a bulk-operation result map keyed by + * `${modelIndex}-${fieldKey}` ids, splitting each id at the FIRST hyphen + * only. Field keys may themselves contain hyphens (a `some-field` column, or + * a nested `profile.some-field` path), so a naive `split('-')` would truncate + * the field key at its first hyphen and silently drop the value during model + * reconstruction. + */ +export function fieldsForModelIndex( + fields: Record, + modelIndex: number, +): Record { + const result: Record = {} + for (const [id, value] of Object.entries(fields)) { + const sep = id.indexOf('-') + if (Number.parseInt(id.slice(0, sep), 10) !== modelIndex) continue + result[id.slice(sep + 1)] = value + } + return result +} diff --git a/packages/stack/src/wasm-inline.ts b/packages/stack/src/wasm-inline.ts index a3409a8f..42d1d75c 100644 --- a/packages/stack/src/wasm-inline.ts +++ b/packages/stack/src/wasm-inline.ts @@ -93,11 +93,23 @@ import { newClient as wasmNewClient, } from '@cipherstash/protect-ffi/wasm-inline' import { resolveIndexType } from '@/encryption/helpers/infer-index-type' +import { + prepareBulkModelsForOperation, + resolveEncryptColumnMap, + setNestedValue, +} from '@/encryption/helpers/model-traversal' import { assertValidNumericValue, assertValueIndexCompatibility, } from '@/encryption/helpers/validation' -import { type AnyV3Table, buildEncryptConfig } from '@/eql/v3' +import { + type AnyV3Table, + buildEncryptConfig, + type V3DecryptedModel, + type V3EncryptedModel, + type V3ModelInput, +} from '@/eql/v3' +import { DATE_LIKE_CASTS } from '@/eql/v3/columns' import { type EncryptionError, EncryptionErrorTypes } from '@/errors' import { type CastAs, @@ -503,9 +515,11 @@ function wasmResult( * result paired with the INPUT index it belongs to. * * Hand-rolled in three places before — the very code the length guard exists - * because it is easy to get subtly wrong. A fourth batch method (the model - * helpers, when they land) could have omitted {@link assertBatchLength} - * without failing a build; routing through here makes that impossible. + * because it is easy to get subtly wrong. (The model helpers do NOT route + * through here: their batches are never sparse — nulls stay in the model — + * and each entry carries `{ modelIndex, fieldKey }` structure instead of a + * positional slot, so they pair results directly and call + * {@link assertBatchLength} themselves.) * * `out` is built with `Array.from`, NOT `items.map(() => null)`: `map` SKIPS * holes in a sparse input, so `bulkDecrypt([a, , b])` would leave index 1 an @@ -554,6 +568,51 @@ function assertBatchLength(op: string, received: number, sent: number): void { } } +/** + * One item of a `decryptBulkFallible` response: the decrypted plaintext, or + * this item's own failure (the batch call itself still resolves). Shared by + * {@link WasmEncryptionClient.bulkDecrypt} and the model decrypt engine. + */ +type FallibleDecryptItem = + | { data: WasmPlaintext } + | { error: string; code?: string } + +/** + * The JS property paths of `table`'s date-like columns (`cast_as: 'date' | + * 'timestamp'`). The model decrypt path rebuilds these into `Date` values: the + * FFI returns date plaintexts in their serialized form (this entry sends them + * as ISO strings — see the model encrypt engine), so a value this client + * encrypted round-trips `Date` → `Date`. Keyed by JS property name (the shape + * the model helpers produce and consume); a row keyed by raw DB column names + * is decrypted but its dates stay ISO strings, since the property→DB map does + * not match them. + */ +function datePropertyPaths(table: AnyV3Table): Set { + const { columns } = table.build() + const propToDb = table.buildColumnKeyMap() + const paths = new Set() + for (const [property, dbName] of Object.entries(propToDb)) { + const castAs = columns[dbName]?.cast_as + if ((DATE_LIKE_CASTS as readonly string[]).includes(castAs as string)) { + paths.add(property) + } + } + return paths +} + +/** + * Rebuild a decrypted plaintext for a model field into a `Date` when the + * column is date-like. An unparseable stored value (a date column written in a + * non-ISO format, or corrupted) would make `new Date(...)` an Invalid Date; + * return the raw value in that case rather than silently handing back an + * Invalid Date whose later `.toISOString()` throws far from here (#742 review). + */ +function reconstructDate(value: WasmPlaintext, isDateColumn: boolean): unknown { + if (!isDateColumn || value == null) return value + const date = new Date(value as string | number) + return Number.isNaN(date.getTime()) ? value : date +} + /** * Internal token used to gate the {@link WasmEncryptionClient} * constructor. Symbols are unique by reference, so external code can't @@ -567,9 +626,10 @@ const INTERNAL_CONSTRUCT = Symbol('cs-wasm-client') * * Wraps an opaque `wasmNewClient` handle and exposes `encrypt`, `decrypt`, * `isEncrypted`, `encryptQuery` / `encryptQueryBulk` for minting v3 query - * terms (#662, which made searchable encryption reachable on the edge), and + * terms (#662, which made searchable encryption reachable on the edge), * `bulkEncrypt` / `bulkDecrypt` for single-round-trip list reads and writes - * (#737). + * (#737), and the model helpers `encryptModel` / `decryptModel` / + * `bulkEncryptModels` / `bulkDecryptModels` (#742). * * ## Every fallible method returns a Result * @@ -589,12 +649,15 @@ const INTERNAL_CONSTRUCT = Symbol('cs-wasm-client') * `isEncrypted` is the one exception, and stays a plain `boolean`: it is a * pure predicate with nothing to fail at, exactly as on the native entry. * - * Still Node-only: the MODEL helpers (`encryptModel` / `decryptModel` and - * their bulk forms). Those are a separate port — this entry has no - * single-model operation to build a bulk one on top of, so adding - * `bulkEncryptModels` alone would be incoherent. Port lazily as Deno / edge - * consumers demand it; the value-level bulk primitives above are what the - * round-trip cost actually hangs on. + * The model helpers run the SAME traversal the native entry uses + * (`@/encryption/helpers/model-traversal` — shared, not ported, so the two + * entries cannot drift on which fields get encrypted), and each call is one + * ZeroKMS round trip regardless of how many fields or models it covers. What + * still differs from the native surface is deliberate and local: failures come + * back as this entry's `{ failure }` Results (rather than a thenable operation + * with `.audit()`), and there is no `.withLockContext()` — identity-bound + * encryption on the edge is configured at client construction via + * `config.authStrategy` instead (#663 context). * * Construct via {@link Encryption} — the constructor is private to * prevent callers from wrapping arbitrary objects in this type. @@ -603,19 +666,57 @@ export class WasmEncryptionClient { /** @internal */ private readonly client: unknown + /** + * The JS-property date paths of every registered table, keyed by table name. + * Precomputed once here (the model decrypt path used to rebuild it per call + * via `table.build()`, which the native typed client precomputes for the same + * reason — #742 review). Also the source of truth for "is this a table the + * client was initialized with": a `decryptModel` against an unknown table is + * a defined failure, matching the native typed client, rather than a silent + * proceed against a foreign schema. + * @internal + */ + private readonly dateFieldsByTable: Map> + /** * @internal Gated by the module-scoped {@link INTERNAL_CONSTRUCT} * symbol: external callers can't obtain it, so {@link Encryption} is * effectively the only constructor. (A `private` constructor would * block {@link Encryption} too, since it lives outside the class.) */ - constructor(token: symbol, client: unknown) { + constructor( + token: symbol, + client: unknown, + schemas: readonly AnyV3Table[] = [], + ) { if (token !== INTERNAL_CONSTRUCT) { throw new Error( '[encryption]: WasmEncryptionClient cannot be constructed directly — use the Encryption() factory.', ) } this.client = client + // Keyed by `tableName` (not object identity) so a table re-imported from + // another module — structurally identical, different reference — still + // resolves, matching how the encrypt config and `build()` key tables. + this.dateFieldsByTable = new Map( + schemas.map((table) => [table.tableName, datePropertyPaths(table)]), + ) + } + + /** + * Resolve a table the client was initialized with to its precomputed date + * paths, or throw the "not initialized with" failure (surfaced as a + * `{ failure }` by the caller's `wasmResult`). Mirrors the native typed + * client's unknown-table guard. + */ + private requireTable(table: AnyV3Table): Set { + const dateFields = this.dateFieldsByTable.get(table.tableName) + if (!dateFields) { + throw new Error( + `[eql/v3]: table "${table.tableName}" is not one this client was initialized with — pass a table given to Encryption({ schemas })`, + ) + } + return dateFields } async encrypt( @@ -624,7 +725,7 @@ export class WasmEncryptionClient { ): Promise> { return wasmResult(async () => { const ffiOpts = { - plaintext, + plaintext: toWasmFfiPlaintext(plaintext), table: opts.table.tableName, column: getColumnName(opts.column), } @@ -855,7 +956,7 @@ export class WasmEncryptionClient { // biome-ignore lint/plugin: the batch crosses the serde boundary, whose shape protect-ffi types as `any` { plaintexts: live.map((item) => ({ - plaintext: item.plaintext, + plaintext: toWasmFfiPlaintext(item.plaintext), table: item.table.tableName, column: getColumnName(item.column), })), @@ -908,13 +1009,9 @@ export class WasmEncryptionClient { ciphertexts: readonly (Encrypted | null | undefined)[], ): Promise>> { return wasmResult(async () => { - type FallibleItem = - | { data: WasmPlaintext } - | { error: string; code?: string } - const { out, placed } = await runBatch< Encrypted, - FallibleItem, + FallibleDecryptItem, WasmPlaintext >( 'bulkDecrypt', @@ -928,7 +1025,7 @@ export class WasmEncryptionClient { { ciphertexts: live.map((ciphertext) => ({ ciphertext })), } as never, - )) as FallibleItem[], + )) as FallibleDecryptItem[], ) // Collect every failure before raising: the FFI already did the work for @@ -958,6 +1055,284 @@ export class WasmEncryptionClient { return out }, EncryptionErrorTypes.DecryptionError) } + + /** + * Encrypt a model's schema-declared fields in ONE ZeroKMS round trip. + * + * Walks `model` against `table`'s columns — matched by JS property name, + * nested fields via a column's dotted path (`'profile.ssn'`) — and encrypts + * exactly the declared fields. Everything else passes through untouched, + * and a `null`/`undefined` schema field is preserved as-is without + * reaching ZeroKMS. The traversal is the native entry's own, shared from + * `@/encryption/helpers/model-traversal` (#742): a column added to the + * schema is picked up by construction, instead of by remembering to extend + * a hand-written `bulkEncrypt` mapping — the failure mode of that mapping + * is a field that silently persists in PLAINTEXT. + * + * `Date` plaintexts (date/timestamp domains) are sent as ISO-8601 strings — + * {@link WasmPlaintext} carries no `Date` across the WASM serde boundary — + * and {@link decryptModel} rebuilds them into `Date` values on the way out. + * + * @example + * ```ts + * const row = await client.encryptModel( + * { id: 1, email: "alice@example.com", verified: true }, + * users, + * ) + * if (row.failure) throw new Error(row.failure.message) + * // row.data = { id: 1, email: , verified: true } + * ``` + */ + async encryptModel< + Table extends AnyV3Table, + T extends Record, + >( + model: V3ModelInput, + table: Table, + ): Promise>> { + return wasmResult(async () => { + const [encrypted] = await this.encryptModelsBatch( + [model as Record], + table, + 'encryptModel', + ) + return encrypted as V3EncryptedModel + }, EncryptionErrorTypes.EncryptionError) + } + + /** + * Encrypt many models in ONE ZeroKMS round trip — {@link encryptModel}'s + * traversal applied per model, with every collected field across every + * model batched into a single FFI call. N models × M columns is still one + * crossing, the same economics as {@link bulkEncrypt}. + * + * The result array is index-aligned with `models`. An empty input returns + * `{ data: [] }` without contacting ZeroKMS. + */ + async bulkEncryptModels< + Table extends AnyV3Table, + T extends Record, + >( + models: Array>, + table: Table, + ): Promise>>> { + return wasmResult( + async () => + (await this.encryptModelsBatch( + models as Record[], + table, + 'bulkEncryptModels', + )) as Array>, + EncryptionErrorTypes.EncryptionError, + ) + } + + /** + * Decrypt every encrypted payload in a model, in ONE ZeroKMS round trip. + * + * Schema-blind on the way in — any value that IS an EQL envelope is + * decrypted, wherever it nests; everything else (nulls included) passes + * through untouched, and the caller's model is never mutated. Schema-aware + * on the way out: `table`'s date-like columns (`date` / `timestamp` domains, + * matched by JS property name) are rebuilt into `Date` values, so a value + * this client encrypted round-trips `Date` → `Date`. `table` must be one the + * client was initialized with, else the call fails (as the native client). + * + * ## Partial failure + * + * Built on the same per-item-fallible primitive as {@link bulkDecrypt}: + * one undecryptable field does not mask the rest. Any failure collapses to + * a single `{ failure }` whose message names EVERY failed field by its + * path in the model (e.g. `profile.ssn`), with its per-item code. + */ + async decryptModel< + Table extends AnyV3Table, + T extends Record, + >(model: T, table: Table): Promise>> { + return wasmResult(async () => { + const [decrypted] = await this.decryptModelsBatch( + [model as Record], + table, + 'decryptModel', + (_modelIndex, fieldKey) => fieldKey, + ) + return decrypted as V3DecryptedModel + }, EncryptionErrorTypes.DecryptionError) + } + + /** + * Decrypt many models in ONE ZeroKMS round trip — {@link decryptModel} + * across a list, which is the shape a page of database rows arrives in. + * The result array is index-aligned with `models`; an empty input returns + * `{ data: [] }` without contacting ZeroKMS. Failures are reported for + * every bad field across the whole batch, labelled `[model ] `. + */ + async bulkDecryptModels< + Table extends AnyV3Table, + T extends Record, + >( + models: T[], + table: Table, + ): Promise>>> { + return wasmResult( + async () => + (await this.decryptModelsBatch( + models as Record[], + table, + 'bulkDecryptModels', + (modelIndex, fieldKey) => `[model ${modelIndex}] ${fieldKey}`, + )) as Array>, + EncryptionErrorTypes.DecryptionError, + ) + } + + /** + * The shared model-encrypt engine behind {@link encryptModel} and + * {@link bulkEncryptModels}: run the shared traversal per model (which builds + * a fresh passthrough tree without touching the caller's model), send every + * collected field in one `encryptBulk` crossing, and set the encrypted + * results back into that tree. + * + * Results pair with fields positionally — `fields` flattens the traversal's + * per-model maps in the order the walk visited them — so the count is + * guarded by {@link assertBatchLength} exactly as the value-level batches + * guard theirs through {@link runBatch}. + */ + private async encryptModelsBatch( + models: Record[], + table: AnyV3Table, + op: string, + ): Promise[]> { + // Empty (or null, from a plain-JS caller) → no round trip, matching the + // native helpers' `if (!models || models.length === 0) return []` (#742). + if (!models || models.length === 0) return [] + this.requireTable(table) + + const { otherFields, operationFields } = prepareBulkModelsForOperation( + models, + table, + ) + const { toColumnName } = resolveEncryptColumnMap(table) + + const fields = operationFields.flatMap((modelFields, modelIndex) => + Object.entries(modelFields).map(([fieldKey, value]) => ({ + modelIndex, + fieldKey, + value, + })), + ) + + let results: Encrypted[] = [] + if (fields.length > 0) { + results = (await wasmEncryptBulk( + // biome-ignore lint/plugin: the FFI handle is an opaque wasm-bindgen pointer with no JS-side type + this.client as never, + // biome-ignore lint/plugin: the batch crosses the serde boundary, whose shape protect-ffi types as `any` + { + plaintexts: fields.map(({ fieldKey, value }) => ({ + // Date → RFC 3339 via the shared serde normalizer (a raw JS Date + // crosses the wasm boundary as `{}`); the decrypt engine rebuilds + // it. Invalid Dates were already rejected per field by the walk. + plaintext: toWasmFfiPlaintext(value), + table: table.tableName, + column: toColumnName(fieldKey), + })), + } as never, + )) as Encrypted[] + assertBatchLength(op, results.length, fields.length) + } + + // `otherFields[modelIndex]` is a fresh tree the walk built with the + // operation fields omitted and nulls kept in place, so the rebuild only + // sets the encrypted results back — no null re-application (which used to + // fabricate phantom nested objects from dotted null keys). + return models.map((_, modelIndex) => { + const rebuilt: Record = { ...otherFields[modelIndex] } + fields.forEach((field, i) => { + if (field.modelIndex !== modelIndex) return + setNestedValue(rebuilt, field.fieldKey.split('.'), results[i]) + }) + return rebuilt + }) + } + + /** + * The shared model-decrypt engine behind {@link decryptModel} and + * {@link bulkDecryptModels}. The traversal runs WITHOUT a table (it + * collects every value that is an encrypted payload — decryption needs no + * schema to find its work); the table drives only the `Date` + * reconstruction. `label` renders a failed field's coordinate for the + * aggregate error, so the single-model caller reports `profile.ssn` while + * the bulk caller reports `[model 2] profile.ssn`. + */ + private async decryptModelsBatch( + models: Record[], + table: AnyV3Table, + op: string, + label: (modelIndex: number, fieldKey: string) => string, + ): Promise[]> { + // Empty (or null) short-circuits BEFORE the table check, so an empty batch + // is `[]` regardless of the table — matching the native helpers (#742). + if (!models || models.length === 0) return [] + const dateFields = this.requireTable(table) + + const { otherFields, operationFields } = + prepareBulkModelsForOperation(models) + + const fields = operationFields.flatMap((modelFields, modelIndex) => + Object.entries(modelFields).map(([fieldKey, value]) => ({ + modelIndex, + fieldKey, + value, + })), + ) + + let results: FallibleDecryptItem[] = [] + if (fields.length > 0) { + results = (await wasmDecryptBulkFallible( + // biome-ignore lint/plugin: the FFI handle is an opaque wasm-bindgen pointer with no JS-side type + this.client as never, + // biome-ignore lint/plugin: the batch crosses the serde boundary, whose shape protect-ffi types as `any` + { + ciphertexts: fields.map(({ value }) => ({ ciphertext: value })), + } as never, + )) as FallibleDecryptItem[] + assertBatchLength(op, results.length, fields.length) + } + + // Same all-failures-at-once contract as bulkDecrypt, labelled by model + // field rather than input index — the caller handed us models, so "which + // field of which model" is the coordinate they can act on. + const failures: string[] = [] + results.forEach((result, i) => { + if ('error' in result) { + const code = result.code ? ` (${result.code})` : '' + const field = fields[i] + failures.push( + ` ${label(field.modelIndex, field.fieldKey)}${code}: ${result.error}`, + ) + } + }) + if (failures.length > 0) { + throw new Error( + `${op} failed for ${failures.length} of ${fields.length} payload(s):\n${failures.join('\n')}`, + ) + } + + return models.map((_, modelIndex) => { + const rebuilt: Record = { ...otherFields[modelIndex] } + fields.forEach((field, i) => { + if (field.modelIndex !== modelIndex) return + const item = results[i] as { data: WasmPlaintext } + setNestedValue( + rebuilt, + field.fieldKey.split('.'), + reconstructDate(item.data, dateFields.has(field.fieldKey)), + ) + }) + return rebuilt + }) + } } /** @@ -1012,8 +1387,9 @@ export async function Encryption( // `INTERNAL_CONSTRUCT` is module-scoped, so this factory is the only // code that can build a `WasmEncryptionClient` — external callers hit - // the constructor guard. - return new WasmEncryptionClient(INTERNAL_CONSTRUCT, client) + // the constructor guard. `schemas` lets the client precompute per-table + // date paths and validate model-op tables against what it was built with. + return new WasmEncryptionClient(INTERNAL_CONSTRUCT, client, schemas) } /** @@ -1080,6 +1456,30 @@ export function getColumnName(col: EncryptOptions['column']): string { ) } +/** + * Normalise a value for the wasm serde boundary, applied at EVERY encrypt/query + * crossing (#742 review — previously only the model path did this). + * + * A JS `Date` has no enumerable own properties, so wasm-bindgen carries it + * across as `{}` — silent corruption of every `date`/`timestamp` column. + * Serialise it to RFC 3339 (which protect-ffi's `parse_naive_date` accepts for + * both casts, yielding the same canonical value as the native Date-object + * path) and reject an invalid `Date` rather than emit `"Invalid Date"`. + * `WasmPlaintext` excludes `Date` so TS callers are already stopped; this is + * the runtime guard for plain-JS callers, the same belt-and-braces rationale + * `assertValidNumericValue` uses on the numeric path. Everything else passes + * through unchanged. + */ +export function toWasmFfiPlaintext(value: unknown): unknown { + if (value instanceof Date) { + if (Number.isNaN(value.getTime())) { + throw new Error('[encryption]: cannot encrypt an invalid Date') + } + return value.toISOString() + } + return value +} + /** * Build the FFI term for one query needle — the ONE place the single and * bulk paths share, so the subtle parts can't drift between them. @@ -1114,7 +1514,7 @@ function toFfiQueryTerm(value: WasmPlaintext, opts: WasmEncryptQueryOptions) { ) assertValueIndexCompatibility(value, indexType, getColumnName(opts.column)) return { - plaintext: value, + plaintext: toWasmFfiPlaintext(value), table: opts.table.tableName, column: getColumnName(opts.column), indexType, diff --git a/skills/stash-encryption/SKILL.md b/skills/stash-encryption/SKILL.md index 6c013d16..506a6043 100644 --- a/skills/stash-encryption/SKILL.md +++ b/skills/stash-encryption/SKILL.md @@ -458,7 +458,19 @@ if (decrypted.failure) throw new Error(decrypted.failure.message) // one ZeroKMS round trip for the whole list, not one per row ``` -`null` / `undefined` entries yield `null` at the same index without reaching ZeroKMS. Because each entry names its own column, one call can cover several columns across many rows. When items fail to decrypt, `failure.message` names every failing index. The model helpers (`encryptModel` / `bulkEncryptModels` / …) are **not** available on the WASM entry. +`null` / `undefined` entries yield `null` at the same index without reaching ZeroKMS. Because each entry names its own column, one call can cover several columns across many rows. When items fail to decrypt, `failure.message` names every failing index. + +**The model helpers are available on the WASM entry too**: `encryptModel(model, table)`, `decryptModel(model, table)`, `bulkEncryptModels(models, table)`, `bulkDecryptModels(models, table)`. They run the same schema walk as the native client — declared columns are encrypted (matched by **JS property name**; nested fields via the column's dotted path), everything else passes through, `null`/`undefined` fields are preserved without reaching ZeroKMS, and the caller's model is never mutated — and a call that touches at least one field is **one ZeroKMS round trip** no matter how many fields or models it covers (an empty batch, or one whose models carry no schema fields, is short-circuited and makes **zero** calls). `table` must be one the client was built with (`Encryption({ schemas })`), else the call fails, as on the native client. `types.Date` / `types.Timestamp` columns round-trip `Date` → `Date` (on the wire they travel as ISO strings); because matching is by JS property name, a row keyed by raw DB column names (e.g. a raw `SELECT` returning `created_on`) still decrypts, but its date fields come back as ISO strings — key your models by the schema's property names. Differences from the native typed client: every method returns a plain `Promise` of the `{ data } | { failure }` Result (no `.audit()` chaining), and there is **no lock-context argument** — identity-bound encryption on the edge is configured at client construction via `config.authStrategy`. Decrypt failures name every failing field: `bulkDecryptModels` prefixes the model index (`[model 1] profile.ssn`), `decryptModel` names the field alone (`profile.ssn`). + +```typescript +const row = await client.encryptModel({ id: 1, email: "alice@example.com" }, users) +if (row.failure) throw new Error(row.failure.message) +// row.data = { id: 1, email: } — only schema columns encrypted + +const back = await client.bulkDecryptModels(encryptedRows, users) +if (back.failure) throw new Error(back.failure.message) +// back.data = plaintext models, index-aligned with the input +``` ## Searchable Encryption