Skip to content

fix(research): persist claim ledgers across resumes - #99

Merged
drewstone merged 8 commits into
mainfrom
fix/persist-claims-and-export-durable-fs
Jul 29, 2026
Merged

fix(research): persist claim ledgers across resumes#99
drewstone merged 8 commits into
mainfrom
fix/persist-claims-and-export-durable-fs

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

Summary

  • persist claims, source support, contradictions, and open questions through the existing knowledge store
  • merge concurrent writers without losing evidence and reject goal, identity, or source-count corruption
  • keep the published FileSystemKbStore string constructor while adding an explicit knowledge-root form
  • export the existing durable filesystem primitives instead of making consumers reimplement crash-safe writes

Proof

  • pnpm lint: 190 files clean
  • pnpm test: 501 passed, 12 skipped across 57 files
  • pnpm build: passed
  • pnpm verify:package: publint, package types, exports, clean-install smoke all passed
  • focused persistence and merge suite: 64 passed across 4 files
  • origin/main merge-tree: clean

…ble-fs

Three defects, each reproduced before it was fixed.

1. Claims could not persist. `createResearchDrivingDriver` tracked every
   claim, its independent-source support, and its contradiction edges in a
   `Map` inside the factory closure with no write path anywhere, so every
   run started empty and all belief state died with the process. Worse,
   `TrackedClaim.supportingHosts` and `.contradicts` were `Set`s, which
   `JSON.stringify` renders as `{}` — a ledger that was stored would have
   been stored with every corroboration count silently zero.

   The record types move to `types.ts` as arrays (dedup enforced by the
   functions that build them, not by the collection), `KbStore` gains
   `putClaimLedger` / `getClaimLedger` / `listClaimLedgers` addressed by a
   per-run id so two runs against one base cannot overwrite each other, and
   `createPersistentResearchDrivingDriver` loads the ledger at construction
   and writes it back after every claim. `ResearchDriver` gains an optional
   `checkpoint()`, which `runVerifiedResearchLoop` calls at the end of each
   round: the deep questions are raised by a synchronous hook, and without
   this the last round's open questions are lost — a resumed run then reports
   itself complete for questions nobody answered.

2. The store was orphaned and forked. `FileSystemKbStore.putIndex` wrote
   `<dir>/index.json` while the reachable `writeKnowledgeIndex` wrote
   `<root>/.agent-knowledge/index.json`: two index writers, two files, and a
   store that reported an empty knowledge base immediately after the indexer
   had filled it. The store is now anchored on the knowledge-base root, keeps
   its records under `.agent-knowledge/` — which is also the directory
   `withKnowledgeMutation` locks — and `writeKnowledgeIndex` writes through
   it, so `index.json` has exactly one writer.

   `putEvent` had no producer. The research loop had always built a
   `research.iteration` event per round and dropped it; it now stores it.
   That event type was missing from `KnowledgeEventSchema`'s hand-restated
   enum, so the one event the package produces would have been rejected on
   the way in. The enum is now derived from `KNOWLEDGE_EVENT_TYPES`, which
   makes that drift unrepresentable.

3. `durable-fs` was package-private. The best low-level primitive here —
   atomic, fsynced, symlink-safe writes through `O_NOFOLLOW` descriptors
   anchored via `/proc/self/fd` — was absent from the entrypoint, so no
   consumer could reach it and the only alternative was a worse copy.

Every guarantee added here has a test that attempts its own violation: a
ledger id that would escape its directory, a second goal merged into an
existing ledger, a second index file, an event type the schema does not
know, a write redirected through a symlink, and a ledger that kept its
claims but lost its questions (which reports the same run complete).
…writing

Persisting a claim ledger is not enough for knowledge to compound. Two writers
reach one ledger — a resumed run beside a live one, or several workers on one
goal — and `putClaimLedger` writes the whole record, so the later write erases
the earlier writer's claims.

- `KbStore.mergeClaimLedger(id, merge)` holds the store's lock across read,
  merge and write. `withKnowledgeMutation` is reentrant, so the nested lock in
  `putClaimLedger` joins the scope rather than deadlocking.
- `claim-ledger.ts` owns the combining rule and claim identity, moved out of the
  driver because a shared durable record's algebra is not one consumer's
  business. Union is sorted, `contested`/`addressed` latch on, `firstSeenRound`
  moves earlier: commutative, associative, idempotent.
- Ledgers for two different goals refuse to merge rather than pooling unrelated
  evidence into one corroboration count.
- The persistent driver merges and rehydrates, so a claim another worker
  corroborated counts toward this driver's completion oracle.
…algebra

`hostOf` decided what counts as an INDEPENDENT source — the rule the whole
corroboration threshold turns on — and was private to the driver, so any other
consumer building a TrackedClaim had to answer it again and would have counted
two pages of one site as independent confirmation. It moves next to claim
identity as `claimSourceHost`.
A contradiction is a property of a pair, and only the refuting writer sees it.
The driver links both ends pairwise as it records; a writer that assembles a
ledger from events has no such moment, and a one-sided edge leaves the original
claim reading as settled. `linkClaimContradictions` is the same rule over a
ledger: idempotent, monotone, and it keeps an edge whose counterpart has not
arrived yet.
tangletools
tangletools previously approved these changes Jul 29, 2026

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — 31bd2613

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-07-29T11:07:59Z

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Value Audit — sound-with-nits

Verdict sound-with-nits
Concerns 2 (2 weak-concern)
Heuristic 0.0s
Duplication 0.0s
Interrogation 146.7s (2 bridge agents)
Total 146.7s

💰 Value — sound-with-nits

Makes research belief state durable + merge-safe across resumes/concurrent workers via a correct monotone merge algebra, fixing a latent Set-serialization bug and a dual index-writer bug along the way; built in the grain of the existing store/lock/durable-fs primitives.

  • What it does: Three things, all toward one end. (1) Extracts the claim-ledger algebra (claim/question identity, independent-source host rule, integrity assertions, and a monotone commutative-associative-idempotent merge) into a new src/claim-ledger.ts. (2) Adds a ClaimLedgerStore capability to KbStore, implemented by FileSystemKbStore (records under .agent-knowledge/claim-ledgers/.json) and MemoryKbStore, w
  • Goals it achieves: A resumed run or two workers researching one goal should compound belief rather than erase it. Before: the driver was in-memory only (lost on crash), and even if persisted, the Set fields would JSON.stringify to {} (silently zeroing every corroboration count), putClaimLedger would whole-record-overwrite (later writer erases earlier writer's claims), and the round event was built and discarded (put
  • Assessment: Sound. The design choices are correct and well-argued. The merge is monotone on every dimension (support only grows, contradictions only accumulate, contested/addressed latch on, firstSeenRound/raisedRound only move earlier) — exactly the CRDT-like property needed for safe replay under a filesystem lock, and the sorted-union rule makes it commutative so arrival order cannot change on-disk bytes. H
  • Better / existing approach: none — this is the right approach. Searched: canonicalizeUrl/hostOf in adaptive-driver.ts, KnowledgeBaseCandidate/claim merging in kb-improvement and optimization, any prior ClaimLedger/mergeClaim usage (none existed pre-PR). The minimal-correct design for crash-safe, concurrent belief accumulation is a commutative merge under a lock, which is what this ships; a single-process serialize would not
  • Model: opencode/zai-coding-plan/glm-5.2
  • Bridge attempts: 2
  • Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content

🎯 Usefulness — sound

Durable claim-ledger persistence plugs straight into the existing research loop and store, fixes three real correctness bugs along the way, and adds no dead or competing surface.

  • Integration: Fully reachable. runVerifiedResearchLoop calls driver.checkpoint?.() at the end of every round (verified-research-loop.ts:346); createPersistentResearchDrivingDriver wires checkpoint to persist() (research-driving-driver.ts:486), which goes through store.mergeClaimLedger. verifySource also persists BEFORE accepting each source (research-driving-driver.ts:437), so a mid-round crash keeps claims eve
  • Fit with existing patterns: In the grain. Ledgers land under /.agent-knowledge/claim-ledgers/.json, the same layout and the same withKnowledgeMutation lock domain the store already owns (docs/architecture.md:40-45). The merge algebra lives in claim-ledger.ts beside the record types rather than inside one driver consumer, matching how schemas/types are separated from I/O elsewhere. The indexer rewrite (indexer.ts:33
  • Real-world viability: Built for concurrency and crashes. mergeClaimLedger holds the mutation lock across read-merge-write (kb-store.ts:354); mergeClaimLedgers is commutative, associative, idempotent with sorted collections, proven by an explicit order-independence test (claim-persistence.test.ts:654). A 6-process concurrent-merge test on separate store instances passes (claim-persistence.test.ts:575). Goal-conflict ref
  • Model: opencode/zai-coding-plan/glm-5.2
  • Bridge attempts: 1

💰 Value Audit

🟡 Third host-extraction function; overlaps private hostOf in adaptive-driver.ts [duplication] ``

claimSourceHost (claim-ledger.ts:51) is a near-duplicate of the private hostOf in adaptive-driver.ts:258 (new URL(uri).hostname.toLowerCase().replace(/^www\./, '')), and lives alongside canonicalizeUrl (adaptive-driver.ts:59) which does full host+path+query canonicalization for dedup. The new function intentionally differs on failure: returns the lowercased raw uri (so offline corpus ids like web/foo count as distinct independent sources) vs hostOf returning ''. That difference is reasoned a

🟡 String-constructor back-compat keeps a second index location alive [maintenance] ``

FileSystemKbStore's string constructor is preserved for published consumers but still writes to

/index.json while the object form writes to /.agent-knowledge/index.json (kb-store.ts:218-223). The dual-writer bug this PR fixes is resolved only for callers using the object form; a string-constructor caller and an object-form caller on the same root still see different index files. This is documented and the object form is the documented canonical one (AGENTS.md, KB_STORE_DIR), so it is a


What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass What it asks
Heuristic Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication Do added function/class names already exist elsewhere in the repo?
Value Audit What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

value-audit · 20260729T111106Z

tangletools
tangletools previously approved these changes Jul 29, 2026

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — 2dc1ed0c

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-07-29T11:28:10Z

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Value Audit — sound-with-nits

Verdict sound-with-nits
Concerns 1 (1 weak-concern)
Heuristic 0.0s
Duplication 0.1s
Interrogation 165.3s (2 bridge agents)
Total 165.4s

💰 Value — sound-with-nits

Makes the research-driving driver's claim ledger (corroboration counts, contradiction edges, open questions) durable and merge-safe across resumes/concurrent writers via a commutative merge algebra, and folds in a real two-index-writers bug fix — a coherent, well-reasoned change with only one minor

  • What it does: Adds a ClaimLedgerStore capability (interface at src/kb-store.ts:73-101) implemented by both MemoryKbStore and FileSystemKbStore, persisting each run's claim ledger as <root>/.agent-knowledge/claim-ledgers/<id>.json (kb-store.ts:312-374). Introduces createPersistentResearchDrivingDriver (research-driving-driver.ts:217) that loads a ledger on construction, merges+rehydrates after every cl
  • Goals it achieves: Belief state that survives kill -9 and resumes: corroboration counts, contradiction edges, and answered-questions persist instead of resetting each round. Concurrent writers on one ledger id (a resumed run beside a live one, or parallel workers on one goal) compound evidence instead of the last writer erasing the others. Ledger bytes are scheduling-independent. Side-goals achieved: close the two
  • Assessment: Strong, in the grain of the codebase. The merge algebra is the correct primitive for the stated failure (putClaimLedger writes the whole record, so two accumulating writers each clobber the other — kb-store.ts:83-96 explains it precisely). mergeClaimLedger holds the single mutation lock across read-merge-write (kb-store.ts:355-374), and I verified withKnowledgeMutation is reentrant via Async
  • Better / existing approach: Searched for an existing record-merge/lock primitive to reuse: mergeTinyChunks, mergeMetrics, mergeScopes, mergeRankedMemoryHits, reconcileInterruptedMemoryPaidCalls are all domain-local, none a generic keyed-record merge-under-lock; KnowledgeFileTransaction (mutation-lock.ts) is for multi-FILE apply/rollback atomicity, a different concern than merging concurrent record accumulations.
  • Model: opencode/zai-coding-plan/glm-5.2
  • Bridge attempts: 2
  • Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content

🎯 Usefulness — sound

A coherent durable-persistence layer for research claim ledgers that fixes two genuine latent integration bugs (a dual index-writer and a schema-enum drift) and is already wired into the shipped verified-research loop.

  • Integration: Reachable and wired, not dead. Two latent integration bugs are fixed and land on real callers: (1) writeKnowledgeIndex (src/indexer.ts:38) previously wrote /.agent-knowledge/index.json while FileSystemKbStore wrote /index.json — confirmed on origin/main (kb-store.ts:145 vs indexer). It now routes through FileSystemKbStore.putIndex, serving cli.ts:86,124,272 and kb-improvement/{transitio
  • Fit with existing patterns: Fits the established grain precisely. KbStore already had put/get/list triples for sources, pages, index, events; ClaimLedgerStore adds the identical triple for ledgers on the SAME two stores, the SAME lock domain (withKnowledgeMutation), and the SAME durable-fs primitives. The merge algebra is factored into its own module (claim-ledger.ts) separate from I/O, matching how the codebase separates pu
  • Real-world viability: Built around the hard case (concurrency + crash), not the happy path. mergeClaimLedger holds the mutation lock across read-merge-write (kb-store.ts:355-374); the combining rule (mergeClaimLedgers) is union-on-support, latch-on-contested/addressed, min-on-firstSeenRound, and sorted outputs — provably commutative, associative, and idempotent (the PR's own tests assert all three algebraic laws plus o
  • Model: opencode/zai-coding-plan/glm-5.2
  • Bridge attempts: 1

💰 Value Audit

🟡 Read-merge-write-under-lock pattern now appears twice in FileSystemKbStore [duplication] ``

updateIndex (kb-store.ts:376-382) and mergeClaimLedger (kb-store.ts:355-374) are the same shape: withKnowledgeMutation(root, read current ?? fallback -> transform -> schema-parse -> writeJsonDurableWithinRoot). A small generic updateJsonRecord<T>(relPath, schema, fallback, merge) helper would DRY ~12 lines and give both paths the lock-spanning guarantee by construction rather than by each call site remembering it. Weak because the two differ in empty-state semantics (index has a syntheti


What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass What it asks
Heuristic Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication Do added function/class names already exist elsewhere in the repo?
Value Audit What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

value-audit · 20260729T113118Z

@drewstone
drewstone merged commit 746e0f0 into main Jul 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants