Skip to content

feat: consolidate TraceDecay V2 PR8-PR13 delivery - #421

Draft
ScriptedAlchemy wants to merge 4698 commits into
masterfrom
codex/tracedecay-total-redesign-plan
Draft

feat: consolidate TraceDecay V2 PR8-PR13 delivery#421
ScriptedAlchemy wants to merge 4698 commits into
masterfrom
codex/tracedecay-total-redesign-plan

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Jul 10, 2026

Copy link
Copy Markdown
Owner

Summary

This draft PR is the consolidated delivery branch for TraceDecay V2 PR8 through PR13.

  • PR8: completed session/LCM temporal retrieval authority.
  • PR9/PR10: code intelligence, exact/lexical/graph retrieval, FastEmbed semantic projection, calibration, and byte-stable fallback.
  • PR11: application, policy, configuration, catalog, Git transaction, and feedback-cycle core.
  • PR12: CLI/MCP/HTTP/LSP production bindings and canonical diagnostics/impact readers.
  • PR13: Hook V2, Context Scout, host lifecycle/bundles, read-only GitHub/CI feedback, and agent proximity.

The branch also contains the supporting rusqlite runtime cutover, worktree-aware incremental indexing, storage retention/Doctor integration, and direct product tests required by the canonical V2 plans.

Verification

  • Focused PR8-PR13 product tests
  • cargo check --all-features
  • cargo test --all-features
  • cargo clippy --workspace --all-targets --all-features -- -D warnings
  • cargo fmt --all -- --check
  • Dashboard and host integration checks
  • Distribution/FastEmbed acceptance
  • Linux, macOS, and Windows CI

Current status: implementation is still being consolidated from the shared checkout and CI is red. This PR remains draft until the working tree is committed, broad verification is stable, and the historical conventional-commit blocker is resolved. It must not be merged in its current state.

Safety boundaries

GitHub review ingestion is read-only. Semantic indexing never blocks exact/lexical/graph retrieval and publishes only complete compatible generations. Local coordination logs, private transcript evidence, secrets, and ad hoc artifacts are excluded from commits.

@changeset-bot

changeset-bot Bot commented Jul 10, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 4615475

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@ScriptedAlchemy ScriptedAlchemy added the WIP Work in progress; do not merge label Jul 10, 2026
@ScriptedAlchemy
ScriptedAlchemy force-pushed the codex/tracedecay-total-redesign-plan branch from e50f8f5 to 4f5ab66 Compare July 10, 2026 05:54
@ScriptedAlchemy

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 986f25cca6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/plans/2026-07-09-tracedecay-brain-rewrite.md Outdated
@ScriptedAlchemy
ScriptedAlchemy force-pushed the codex/tracedecay-total-redesign-plan branch 4 times, most recently from 5862bd6 to 53502de Compare July 11, 2026 16:52
@ScriptedAlchemy
ScriptedAlchemy force-pushed the codex/tracedecay-total-redesign-plan branch 4 times, most recently from 642a33a to 5f2f610 Compare July 23, 2026 17:06
@ScriptedAlchemy ScriptedAlchemy changed the title [WIP] Plan TraceDecay V2 brain rewrite feat: consolidate TraceDecay V2 PR8-PR13 delivery Jul 24, 2026
@ScriptedAlchemy
ScriptedAlchemy force-pushed the codex/tracedecay-total-redesign-plan branch from 76dee0f to 0258866 Compare July 24, 2026 17:16
ScriptedAlchemy added a commit that referenced this pull request Jul 27, 2026
Record the 2026-07-27 audit in the authoritative V2 status files.

Three delivered claims (Plan 16 worktree/store identity, Plan 38 cursor
repair, Plans 18/23 end-to-end sanitization) lost their tests in
9e3ca9f. Filed as verification-coverage corrections, not retractions:
the assertion is missing, the behavior is not withdrawn.

Also record the six instances of gates attesting to something they never
checked, the libtest empty-filter asymmetry that hides them, the six
product defects fixed tonight with their SHAs, and Plan 11's corrected
open-gap list including two items the widened accessibility matrix
surfaced.

None of tonight's commits has CI validation: PR #421 has been
conflicting since 05:13 UTC, so no CI has run since 01:24 UTC. Every
fix is recorded as locally verified only.
@ScriptedAlchemy
ScriptedAlchemy force-pushed the codex/tracedecay-total-redesign-plan branch from 062285a to b73f8bf Compare August 1, 2026 18:32
ScriptedAlchemy and others added 13 commits August 2, 2026 01:26
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The first query against a restored generation paid the whole cold path:
the sealed decode (which re-mints every file's exact-extraction authority
with a canonical SHA-256 over every chunk, then repeats the full canonical
validation sweep), plus the exact-admission sweep, record lookup indices,
lane owners, and test-attribution join — all deferred behind OnceLocks and
charged to whichever request happened to arrive first. That is O(store) on
the serving path, violating the invariant in docs/SERVING-PATH-PERFORMANCE.

Activation now owns that work:

- mount runs the worktree open on the blocking pool instead of an async
  runtime worker, and warms every per-generation derivation there;
- reconcile completion warms the generation it just published;
- the publication store gains a single-flight decode barrier. The cache
  lock is never held across a decode, concurrent callers for the same
  generation share one decode instead of duplicating it, and a publication
  landing mid-decode retires the superseded result via an epoch guard;
- the active generation is pinned outside the LRU, so cursor traffic over
  superseded generations can never evict it.

Fail-closed is unchanged: only success is memoized, so a corrupt store
still runs the complete check and errors on every request, and warming is
best-effort — it is a pre-warm, never a gate.

Measured on a release build over a 1.4K-file fixture: first-query cost
3.57s -> 88ms.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cold generation decode spent 40%+ of its cost in
ExactExtractionAuthorityV1 mint/restore: every chunk was serialized to
canonical JSON and SHA-256'd one at a time on one thread.

Fan the per-chunk sweeps (mint, validate_all, admit_all, and the
per-chunk leg of CodeFileChunksV1::validate) across the pool above a
16-chunk threshold. Digests are independent, results are collected in
chunk order, and the reported failure is still the lowest-index one, so
callers observe exactly what the sequential short-circuit produced.

Cut the allocation churn inside canonical writing: strings and object
keys now stream their JSON escapes straight into the sink instead of
allocating a String per value via serde_json::to_string, integers format
into a stack buffer, and already-sorted objects skip the collect-and-sort
Vec. Output stays byte-identical, covered by escape-class and number
equivalence tests against the rendering they replace.

Measured on a 1201-chunk fixture (release, min of 25):
mint 10.98ms -> 2.0ms; the canonical change alone takes the
single-threaded sweep 10.98ms -> 9.6ms.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`tracedecay_runtime_core::ports::registered_schema` fails closed until the
composition root installs the real schema, so every test process that opened a
profile- or session-scoped shard without registering died with "no registered
global/session schema installer is registered". That was the largest CI cluster:
~240 lib tests across six crates.

Register the real installer at each crate's *fixture funnel* rather than at
every test, following the precedent already set by `tests/common/mod.rs` and
`src/dashboard.rs`:

- `tracedecay-global-db` self-registers inside `open_registered_test_database`,
  `RegisteredGlobalDbHarness::open`, both harness `open` constructors, and the
  LCM schema read-only fixture. That covers every downstream crate that builds
  fixtures on `HostAdmissionTestRuntimeV1` / `RegisteredGlobalDbTestRuntime`.
- `tracedecay-migrate`, `tracedecay-usecases`, `tracedecay-dashboard-api`, and
  the root crate register at the handful of fixtures that reach
  `Database::publish_test_runtime` directly.
- `tracedecay-runtime-core` sits below the crate that owns the schema and can
  never install it, so its *own* `cfg(test)` build treats an unregistered port
  as an empty profile sidecar. Kernel fixtures only ever exercise the graph
  shard beside it. No other build reaches that arm.

Two adjacent fixture defects surfaced once the port stopped short-circuiting,
both in `open_registered_test_database`: schema DDL ran on `Database::conn()`
(the retained *reader*, so the migration SQL channel answered
`WriterUnavailable`), and the mode was hard-coded to `Initialize` even for a
store an earlier stage of the same test had already materialised. Both fixed.

Production stays fail-closed, and
`crates/tracedecay-global-db/tests/registered_schema_fail_closed.rs` now proves
it from a process that links the kernel as an ordinary dependency and registers
nothing.

Lib tests, `--all-features`: 305 -> 157 failing (148 fixed, 0 newly failing);
the schema-installer error is gone from every measured crate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`HostBundleError::StorageFailure` had ~115 construction sites across the
host bundle lifecycle that all rendered the identical sentence, so a
failed install or uninstall reported nothing an operator or a bug report
could act on.

Carry the raising site in the variant. `host_bundle_storage_failure!()`
stamps `file:line` at the call site, keeping the payload a `&'static str`
so the enum stays `Copy`/`Eq`. Tests that assert the whole error value
now compare against a named sentinel instead of a second macro
expansion, which would carry the assertion's own line.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`tracedecay uninstall --agent codex` failed with a bare storage failure
and rolled the whole transaction back, so uninstall was a no-op that
reported an error.

Codex's `host_component_registration` gated on the deployed plugin
*source* bundle: with `.codex-plugin/plugin.json` on disk it never
returned `Missing`, only `Current` or `Repairable`. But that bundle is an
artifact-layer deployment which `deactivate_deployed_host_registration`
deliberately leaves behind, so after a correct deactivation the component
still read `Repairable` while the uninstall verify demands `Missing`.

Gate on the registration surface instead, matching Claude's residue
contract: the `[hooks.state]` trust keys, `[mcp_servers.tracedecay]`, the
`[plugins."tracedecay@..."]` activation records, and the personal
marketplace entry — exactly what deactivation clears. A staged but never
activated bundle still reads `Repairable`, since its marketplace entry
survives.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ScriptedAlchemy and others added 30 commits August 3, 2026 00:05
The three projection failures blamed on the migration removal trace to
the earlier recall fix that narrowed derived logical copies to genuine
re-emissions: a parent-message link is threading, not a copy. The tests
encoded the old every-reply-is-a-copy semantics and are re-contracted;
copy persistence, restart idempotency, and back-off coverage are
unchanged in strength. The retrieval candidate fixture had aborted at
seed time since its FK on session_turns went unsatisfied, so its
expectations had never run; it now seeds parent turns, scopes the
snapshot to the fixture session, asserts all in-window occurrences, and
narrows three query-plan assertions that were unsatisfiable by query
shape. One assertion stays red on purpose: the root-wide derived
candidate query full-scans session_derived_evidence because its
covering index leads with session_id - a real performance question, not
a test defect.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts:
#	crates/tracedecay-migrate/src/manifest/runtime.rs
#	tests/storage_suite/migration_manifest_test.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

WIP Work in progress; do not merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant