fix(research): persist claim ledgers across resumes - #99
Conversation
…ble-fs
Three defects, each reproduced before it was fixed.
1. Claims could not persist. `createResearchDrivingDriver` tracked every
claim, its independent-source support, and its contradiction edges in a
`Map` inside the factory closure with no write path anywhere, so every
run started empty and all belief state died with the process. Worse,
`TrackedClaim.supportingHosts` and `.contradicts` were `Set`s, which
`JSON.stringify` renders as `{}` — a ledger that was stored would have
been stored with every corroboration count silently zero.
The record types move to `types.ts` as arrays (dedup enforced by the
functions that build them, not by the collection), `KbStore` gains
`putClaimLedger` / `getClaimLedger` / `listClaimLedgers` addressed by a
per-run id so two runs against one base cannot overwrite each other, and
`createPersistentResearchDrivingDriver` loads the ledger at construction
and writes it back after every claim. `ResearchDriver` gains an optional
`checkpoint()`, which `runVerifiedResearchLoop` calls at the end of each
round: the deep questions are raised by a synchronous hook, and without
this the last round's open questions are lost — a resumed run then reports
itself complete for questions nobody answered.
2. The store was orphaned and forked. `FileSystemKbStore.putIndex` wrote
`<dir>/index.json` while the reachable `writeKnowledgeIndex` wrote
`<root>/.agent-knowledge/index.json`: two index writers, two files, and a
store that reported an empty knowledge base immediately after the indexer
had filled it. The store is now anchored on the knowledge-base root, keeps
its records under `.agent-knowledge/` — which is also the directory
`withKnowledgeMutation` locks — and `writeKnowledgeIndex` writes through
it, so `index.json` has exactly one writer.
`putEvent` had no producer. The research loop had always built a
`research.iteration` event per round and dropped it; it now stores it.
That event type was missing from `KnowledgeEventSchema`'s hand-restated
enum, so the one event the package produces would have been rejected on
the way in. The enum is now derived from `KNOWLEDGE_EVENT_TYPES`, which
makes that drift unrepresentable.
3. `durable-fs` was package-private. The best low-level primitive here —
atomic, fsynced, symlink-safe writes through `O_NOFOLLOW` descriptors
anchored via `/proc/self/fd` — was absent from the entrypoint, so no
consumer could reach it and the only alternative was a worse copy.
Every guarantee added here has a test that attempts its own violation: a
ledger id that would escape its directory, a second goal merged into an
existing ledger, a second index file, an event type the schema does not
know, a write redirected through a symlink, and a ledger that kept its
claims but lost its questions (which reports the same run complete).
…writing Persisting a claim ledger is not enough for knowledge to compound. Two writers reach one ledger — a resumed run beside a live one, or several workers on one goal — and `putClaimLedger` writes the whole record, so the later write erases the earlier writer's claims. - `KbStore.mergeClaimLedger(id, merge)` holds the store's lock across read, merge and write. `withKnowledgeMutation` is reentrant, so the nested lock in `putClaimLedger` joins the scope rather than deadlocking. - `claim-ledger.ts` owns the combining rule and claim identity, moved out of the driver because a shared durable record's algebra is not one consumer's business. Union is sorted, `contested`/`addressed` latch on, `firstSeenRound` moves earlier: commutative, associative, idempotent. - Ledgers for two different goals refuse to merge rather than pooling unrelated evidence into one corroboration count. - The persistent driver merges and rehydrates, so a claim another worker corroborated counts toward this driver's completion oracle.
…algebra `hostOf` decided what counts as an INDEPENDENT source — the rule the whole corroboration threshold turns on — and was private to the driver, so any other consumer building a TrackedClaim had to answer it again and would have counted two pages of one site as independent confirmation. It moves next to claim identity as `claimSourceHost`.
A contradiction is a property of a pair, and only the refuting writer sees it. The driver links both ends pairwise as it records; a writer that assembles a ledger from events has no such moment, and a one-sided edge leaves the original claim reading as settled. `linkClaimContradictions` is the same rule over a ledger: idempotent, monotone, and it keeps an edge whose counterpart has not arrived yet.
…d-export-durable-fs
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — 31bd2613
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-29T11:07:59Z
tangletools
left a comment
There was a problem hiding this comment.
🟡 Value Audit — sound-with-nits
| Verdict | sound-with-nits |
| Concerns | 2 (2 weak-concern) |
| Heuristic | 0.0s |
| Duplication | 0.0s |
| Interrogation | 146.7s (2 bridge agents) |
| Total | 146.7s |
💰 Value — sound-with-nits
Makes research belief state durable + merge-safe across resumes/concurrent workers via a correct monotone merge algebra, fixing a latent Set-serialization bug and a dual index-writer bug along the way; built in the grain of the existing store/lock/durable-fs primitives.
- What it does: Three things, all toward one end. (1) Extracts the claim-ledger algebra (claim/question identity, independent-source host rule, integrity assertions, and a monotone commutative-associative-idempotent merge) into a new src/claim-ledger.ts. (2) Adds a ClaimLedgerStore capability to KbStore, implemented by FileSystemKbStore (records under .agent-knowledge/claim-ledgers/.json) and MemoryKbStore, w
- Goals it achieves: A resumed run or two workers researching one goal should compound belief rather than erase it. Before: the driver was in-memory only (lost on crash), and even if persisted, the Set fields would JSON.stringify to {} (silently zeroing every corroboration count), putClaimLedger would whole-record-overwrite (later writer erases earlier writer's claims), and the round event was built and discarded (put
- Assessment: Sound. The design choices are correct and well-argued. The merge is monotone on every dimension (support only grows, contradictions only accumulate, contested/addressed latch on, firstSeenRound/raisedRound only move earlier) — exactly the CRDT-like property needed for safe replay under a filesystem lock, and the sorted-union rule makes it commutative so arrival order cannot change on-disk bytes. H
- Better / existing approach: none — this is the right approach. Searched: canonicalizeUrl/hostOf in adaptive-driver.ts, KnowledgeBaseCandidate/claim merging in kb-improvement and optimization, any prior ClaimLedger/mergeClaim usage (none existed pre-PR). The minimal-correct design for crash-safe, concurrent belief accumulation is a commutative merge under a lock, which is what this ships; a single-process serialize would not
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 2
- Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content
🎯 Usefulness — sound
Durable claim-ledger persistence plugs straight into the existing research loop and store, fixes three real correctness bugs along the way, and adds no dead or competing surface.
- Integration: Fully reachable. runVerifiedResearchLoop calls driver.checkpoint?.() at the end of every round (verified-research-loop.ts:346); createPersistentResearchDrivingDriver wires checkpoint to persist() (research-driving-driver.ts:486), which goes through store.mergeClaimLedger. verifySource also persists BEFORE accepting each source (research-driving-driver.ts:437), so a mid-round crash keeps claims eve
- Fit with existing patterns: In the grain. Ledgers land under /.agent-knowledge/claim-ledgers/.json, the same layout and the same withKnowledgeMutation lock domain the store already owns (docs/architecture.md:40-45). The merge algebra lives in claim-ledger.ts beside the record types rather than inside one driver consumer, matching how schemas/types are separated from I/O elsewhere. The indexer rewrite (indexer.ts:33
- Real-world viability: Built for concurrency and crashes. mergeClaimLedger holds the mutation lock across read-merge-write (kb-store.ts:354); mergeClaimLedgers is commutative, associative, idempotent with sorted collections, proven by an explicit order-independence test (claim-persistence.test.ts:654). A 6-process concurrent-merge test on separate store instances passes (claim-persistence.test.ts:575). Goal-conflict ref
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
💰 Value Audit
🟡 Third host-extraction function; overlaps private hostOf in adaptive-driver.ts [duplication] ``
claimSourceHost (claim-ledger.ts:51) is a near-duplicate of the private hostOf in adaptive-driver.ts:258 (
new URL(uri).hostname.toLowerCase().replace(/^www\./, '')), and lives alongside canonicalizeUrl (adaptive-driver.ts:59) which does full host+path+query canonicalization for dedup. The new function intentionally differs on failure: returns the lowercased raw uri (so offline corpus ids likeweb/foocount as distinct independent sources) vs hostOf returning ''. That difference is reasoned a
🟡 String-constructor back-compat keeps a second index location alive [maintenance] ``
FileSystemKbStore's string constructor is preserved for published consumers but still writes to
/index.json while the object form writes to /.agent-knowledge/index.json (kb-store.ts:218-223). The dual-writer bug this PR fixes is resolved only for callers using the object form; a string-constructor caller and an object-form caller on the same root still see different index files. This is documented and the object form is the documented canonical one (AGENTS.md, KB_STORE_DIR), so it is a
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — 2dc1ed0c
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-29T11:28:10Z
tangletools
left a comment
There was a problem hiding this comment.
🟡 Value Audit — sound-with-nits
| Verdict | sound-with-nits |
| Concerns | 1 (1 weak-concern) |
| Heuristic | 0.0s |
| Duplication | 0.1s |
| Interrogation | 165.3s (2 bridge agents) |
| Total | 165.4s |
💰 Value — sound-with-nits
Makes the research-driving driver's claim ledger (corroboration counts, contradiction edges, open questions) durable and merge-safe across resumes/concurrent writers via a commutative merge algebra, and folds in a real two-index-writers bug fix — a coherent, well-reasoned change with only one minor
- What it does: Adds a
ClaimLedgerStorecapability (interface at src/kb-store.ts:73-101) implemented by bothMemoryKbStoreandFileSystemKbStore, persisting each run's claim ledger as<root>/.agent-knowledge/claim-ledgers/<id>.json(kb-store.ts:312-374). IntroducescreatePersistentResearchDrivingDriver(research-driving-driver.ts:217) that loads a ledger on construction, merges+rehydrates after every cl - Goals it achieves: Belief state that survives
kill -9and resumes: corroboration counts, contradiction edges, and answered-questions persist instead of resetting each round. Concurrent writers on one ledger id (a resumed run beside a live one, or parallel workers on one goal) compound evidence instead of the last writer erasing the others. Ledger bytes are scheduling-independent. Side-goals achieved: close the two - Assessment: Strong, in the grain of the codebase. The merge algebra is the correct primitive for the stated failure (
putClaimLedgerwrites the whole record, so two accumulating writers each clobber the other — kb-store.ts:83-96 explains it precisely).mergeClaimLedgerholds the single mutation lock across read-merge-write (kb-store.ts:355-374), and I verifiedwithKnowledgeMutationis reentrant via Async - Better / existing approach: Searched for an existing record-merge/lock primitive to reuse:
mergeTinyChunks,mergeMetrics,mergeScopes,mergeRankedMemoryHits,reconcileInterruptedMemoryPaidCallsare all domain-local, none a generic keyed-record merge-under-lock;KnowledgeFileTransaction(mutation-lock.ts) is for multi-FILE apply/rollback atomicity, a different concern than merging concurrent record accumulations. - Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 2
- Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content
🎯 Usefulness — sound
A coherent durable-persistence layer for research claim ledgers that fixes two genuine latent integration bugs (a dual index-writer and a schema-enum drift) and is already wired into the shipped verified-research loop.
- Integration: Reachable and wired, not dead. Two latent integration bugs are fixed and land on real callers: (1) writeKnowledgeIndex (src/indexer.ts:38) previously wrote /.agent-knowledge/index.json while FileSystemKbStore wrote /index.json — confirmed on origin/main (kb-store.ts:145 vs indexer). It now routes through FileSystemKbStore.putIndex, serving cli.ts:86,124,272 and kb-improvement/{transitio
- Fit with existing patterns: Fits the established grain precisely. KbStore already had put/get/list triples for sources, pages, index, events; ClaimLedgerStore adds the identical triple for ledgers on the SAME two stores, the SAME lock domain (withKnowledgeMutation), and the SAME durable-fs primitives. The merge algebra is factored into its own module (claim-ledger.ts) separate from I/O, matching how the codebase separates pu
- Real-world viability: Built around the hard case (concurrency + crash), not the happy path. mergeClaimLedger holds the mutation lock across read-merge-write (kb-store.ts:355-374); the combining rule (mergeClaimLedgers) is union-on-support, latch-on-contested/addressed, min-on-firstSeenRound, and sorted outputs — provably commutative, associative, and idempotent (the PR's own tests assert all three algebraic laws plus o
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
💰 Value Audit
🟡 Read-merge-write-under-lock pattern now appears twice in FileSystemKbStore [duplication] ``
updateIndex(kb-store.ts:376-382) andmergeClaimLedger(kb-store.ts:355-374) are the same shape:withKnowledgeMutation(root, read current ?? fallback -> transform -> schema-parse -> writeJsonDurableWithinRoot). A small genericupdateJsonRecord<T>(relPath, schema, fallback, merge)helper would DRY ~12 lines and give both paths the lock-spanning guarantee by construction rather than by each call site remembering it. Weak because the two differ in empty-state semantics (index has a syntheti
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
Summary
Proof