feat(sync): declarative derive fields + real identity-key preview (v1.55.0)

[siddharth-bhansali]

Closes #129 · INT-4408

Ships the last two items on #129 — the ones I'd deferred out of #181.

1. derive — flat fields without a shell

#129 asked for the gmail from_email extraction to move out of a user-editable gmail-extract.jq and into the built-in profile.

It could not go in as a transform. transform is the only existing way to produce a derived field, and it runs as spawn('sh', ['-c', cmd]) — so a built-in relying on it would require jq on every user's PATH, and on Windows would ENOENT and silently resolve null, leaving the sync reporting success with the field absent. Not something to ship in a default profile.

So this adds a declarative alternative:

"derive": {
  "from_email": {
    "path": "messages[0].payload.headers[name=From].value",
    "extract": "email"
  },
  "company": "organization.name"
}

• Reuses the identityKeys path resolver — [] wildcards, [0] indexes, [name=From] filters. One path syntax for profile authors, not two.
• extract: "email" applies the same address extraction identityKeys uses for email-prefixed values, so a display-name header yields jane@acme.com.
• A path resolving to nothing omits the field rather than writing null — a null would break --where filters and leave every record carrying dead keys.
• A path resolving to several values takes the first. It's a flat field by definition, and the path syntax is expressive enough to say which one you meant.

Applied inside writePageToMemory — deliberately, because that's the one point both sync phases pass through. Phase 1 writes list-shape records and phase 2 writes enriched ones; a derived field applied in only one would flicker between runs.

Shipped on profiles/gmail/gmailThreads.json, which is #129's actual ask.

2. sync test previews what actually resolves

Previously, for an enriching profile, the preview ran against list-shape samples where the participant paths don't exist yet — so it reported zero keys plus a resolvesAfterEnrich note. Accurate, but it never showed the author what would resolve, which is what acceptance criterion 4 wanted.

It now spends one detail call on the first sample and previews the merged shape, flagged previewedAfterEnrich. If that call fails (rate limit, permissions, an unreachable detail action) it falls back to the previous behaviour, so the command never gets worse.

The fetch reuses enrichSingleRow and the same resultsPath / fields / exclude / merge handling as the real phase, exported as enrichOneForPreview. A preview that constructed its own request would drift from what a sync actually writes, which is worse than no preview.

Worth a reviewer's opinion: this makes sync test perform a real (read-only) API call for enriching profiles, where before it was purely a dry check. That's the reading closest to the acceptance criterion, but if you'd rather it were behind --deep it's a small change.

Tests

24 new, 457 total, 0 skipped, all passing.

• derive.test.ts — resolver edge cases: fan-out, missing paths, non-scalar values, email extraction with and without a display name, the built-in gmail profile, and an assertion that no built-in requires a shell.
• mem-writer.test.ts — an integration test against live PGlite proving derived fields reach the stored record and that a missing path writes nothing. The wiring is where a silent no-op would hide, so it's covered separately from the resolver.

Notes

• Version 1.55.0. Sits on top of fix(deps): bump better-sqlite3 to v12 for Node 24 support (v1.54.1) #184 (1.54.1); merge that first or this one needs a version rebase.
• Docs updated in guide-content.ts (new "Derived fields" section placed before Record Transform, since it's what you should reach for first) and skills/one/SKILL.md. No CLI help change — derive is a profile field, not a flag.

[linear]

INT-4408