feat(tables): add automatic row expiration with TTL - #7071
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
Greptile SummaryThis PR adds a TTL table-column type backed by Unix epoch seconds, including validation, timezone-aware editing, import and conversion support, and documentation.
Confidence Score: 4/5The PR appears safe to merge, with a non-blocking cleanup-fairness issue that can leave later tables behind under sustained high-volume expiration workloads. The TTL representation and mutation paths are internally consistent, but the cleanup worker's fixed ordering and shared batch budget allow an early table to repeatedly monopolize scheduled cleanup capacity. Files Needing Attention: apps/sim/background/cleanup-table-row-ttl.ts
|
| Filename | Overview |
|---|---|
| apps/sim/background/cleanup-table-row-ttl.ts | Implements locked, batched TTL deletion; the global budget and fixed table ordering can create unbounded lag under sustained backlog. |
| apps/sim/lib/table/column-types/ttl.ts | Defines TTL epoch-second coercion, validation, conversion, and timezone-aware formatting. |
| apps/sim/lib/table/columns/service.ts | Applies source-aware value conversion and validates the resulting schema before retyping columns. |
| apps/sim/lib/table/schema-invariants.ts | Extends shared schema invariants with registry-declared per-table column limits. |
| apps/sim/app/api/cron/cleanup-table-row-ttl/route.ts | Adds an authenticated cron endpoint that dispatches one deduplicated, serialized cleanup job. |
| apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/table-grid/cells/inline-editors.tsx | Normalizes date-editor saves through timezone-aware column coercion so TTL cells persist epoch seconds. |
| helm/sim/values.yaml | Enables the TTL cleanup endpoint on a five-minute non-overlapping cron schedule. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
Cron[Five-minute cron] --> Route[Authenticated cleanup route]
Route --> Queue[Serialized cleanup job]
Queue --> List[List tables with expired TTL rows]
List --> Lock[Acquire table advisory lock]
Lock --> Batch[Delete up to 500 expired rows]
Batch -->|More rows and budget remains| Lock
Batch -->|Table complete| Signal[Signal table rows changed]
Signal -->|More tables and budget remains| Lock
Reviews (1): Last reviewed commit: "feat(tables): add row TTL expiration" | Re-trigger Greptile
There was a problem hiding this comment.
Review completed against the latest diff
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
9a22284 to
0240ade
Compare
0240ade to
93d65ea
Compare
b604ffa to
f00f927
Compare
f00f927 to
dc9903c
Compare
dc9903c to
c99a841
Compare
2763e53 to
c995a4a
Compare
c995a4a to
8d6b2e8
Compare
Page cleanup by created_at and id, with a supporting index, so capped runs make progress on the oldest rows first.
8d6b2e8 to
e650905
Compare
Summary
Tables can now expire rows automatically with one Expiration column. Cells use the date editor, APIs and workflows store integer Unix epoch seconds, and a bounded cleanup job removes eligible rows every fifteen minutes.
This is PR 2 of 4 in the row-expiration stack. It builds on the generic registry hooks in PR #7119; PR #7072 supplies the safety-critical timezone consolidation; PR #7161 lets deletions trigger workflows.
The global
table-row-ttlruntime feature flag gates UI availability, schema mutations, and cleanup dispatch. Outside AppConfig, theTABLE_ROW_TTLsecret is the global on/off fallback.Cleanup is round-robin across candidate tables and is capped at 100 batches of 500 rows per run. Each table is scanned by
created_atandidso older rows cannot starve, but only the Expiration value decides whether a row is deleted. The supporting database index matches that scan order.Type of Change
Testing
bun run type-check, lint, all 33 repository audits, the block-registry check, and the docs-manifest check pass at the stack tip.bun run check:migrations origin/stagingpasses. A local migration apply could not run because no local PostgreSQL server was available.Checklist
Screenshots/Videos
No screenshots captured. Component tests cover the Expiration picker, its one-column limit, disabled-state explanation, and cell rendering.