feat(kb): hygiene release v0.6.4 — slug data loss, audit-kb silent failure, format contract

.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "axme-code",
-  "version": "0.6.3",
+  "version": "0.6.4",
   "description": "(Alpha) Persistent memory, architectural decisions, and safety guardrails for Claude Code. Your agent starts every session with full project context — stack, decisions, patterns, safety rules, and a handoff from the previous session.",
   "author": {
     "name": "AXME AI",

CHANGELOG.md

@@ -2,6 +2,43 @@
 
 ## [Unreleased]
 
+## [0.6.4] - 2026-08-18
+
+Knowledge-base hygiene release. A full manual compaction of a production knowledge base (282 memories / 115 decisions, ~192k tokens at session start) surfaced one data-loss bug, one silent-failure bug, and a format contract that was documented everywhere and enforced nowhere. This release fixes the bugs, makes the contract measurable, and gives agents the tools to clean up without bypassing MCP.
+
+### Fixed
+
+- **Silent data loss: a non-Latin title produced an empty slug.** `toSlug` / `toMemorySlug` were `text.toLowerCase().replace(/[^a-z0-9]+/g, "-")`, which maps any fully non-Latin title to `""`. The memory was written as bare `.md` — a dotfile, invisible to `ls` and to every shell glob over `memory/*/*.md` — and the **next** such title overwrote it. Four affected files were found on one project, with two confirmed overwrites. Slug generation now transliterates Cyrillic and Greek, strips Latin diacritics, falls back to a content hash when nothing survives, and prefixes digit-only slugs (`16-07` → `memory-16-07`) which were useless for semantic search. `saveMemory` additionally refuses to overwrite a file whose stored title differs from the incoming one, writing a suffixed slug and reporting the collision instead. Same title still overwrites — that is how an entry is revised.
+- **`audit-kb` could exit 0 having written nothing.** A real run analysed 109 decisions correctly for four minutes, reached a conclusion, wrote zero bytes, and reported success; the result counters were `resultText.match(/supersed/gi).length`, i.e. word-counting the agent's prose. The command now snapshots every entry's loaded-layer size before and after the agent runs and reports the measured diff. A pass that changed nothing prints `NO CHANGES WRITTEN`, explains that this is a failed pass rather than a clean base, and exits 2.
+- **First page of paginated output could be empty.** `paginateSections` never split a section larger than the page limit, so a caller passing `["## Project Memories", <60KB block>]` produced a page 1 containing only the heading with all content on page 2. Oversized sections are now split along line boundaries; no content is lost or truncated.
+- **Leaked tool-call markup was persisted verbatim.** Three records ended with `</description><parameter name="keywords">[...]` — a malformed client emission gluing one argument's XML frame onto another's text. Storage now strips everything from the first stray frame tag onward on write, and reports which fields it cleaned so the agent can verify what landed.
+- **Repairing leaked markup could delete the deferred section.** Found while validating the repair against the two affected records in axme-code's own base: the leak sits at the *end* of the description, with `## Details` directly below it, so a cut-to-everything-after-the-tag repair would have destroyed the very layer this release exists to protect. Field-level stripping still cuts to the end (a single argument value has no legitimate tail), but record-level repair now truncates the offending line, drops only the frame-only lines that follow, and keeps everything from the first real line onward.
+- **Frontmatter rewrites could delete the following field.** The key-replacement regex used `\s*`, which matches newlines, so rewriting an empty `slug:` consumed the `type:` line under it and made the record unparseable. Now matched with horizontal whitespace only.
+
+### Added
+
+- **`axme-code kb-doctor [path] [--fix]`** — deterministic storage-defect scan. No LLM, no network, milliseconds on a 300-entry base. Finds empty and degenerate slugs, leaked tool-call markup, frontmatter that disagrees with the filename, entries whose loaded layer overruns the catalog budget, and duplicate titles. `--fix` repairs the mechanical ones (renames preserve content and correct the frontmatter); overlong and duplicate entries need judgment and are reported only. Exits 1 on outstanding defects, so it works as a CI gate. Also exposed as the `axme_kb_doctor` MCP tool.
+- **`axme_archive_memory` / `axme_archive_decision`** — the missing half of the storage API. axme-code could create knowledge but not retire it, while the knowledge bases it builds carry a rule of their own ("write to axme-code storage via MCP tools only, never manually"), leaving an agent asked to clean up with no legal move; the first real compaction had to bypass MCP with file operations. Archival moves entries to `.axme-code/archive/` preserving structure, stamps the reason into the frontmatter, and marks decisions `superseded`/`revoked` before the move. Nothing is deleted; an archival is undone by moving one file back. A `superseded_by` that does not resolve is refused rather than written as a dangling pointer.
+- **`catalog.excerpt_chars` and `catalog.size_warn` in `config.yaml`.** The catalog excerpt width was a hardcoded `slice(0, 200)` mentioned in no documentation. Authoring 200+ entries against an undocumented constant in someone else's tool is a bet that it never moves; it is now configurable, clamped to a sane range on write, readable via `axme-code config get catalog.excerpt_chars`, and named in the `axme_save_memory` tool description.
+- **`audit-kb --dry-run`** — preview a compaction pass. The agent reads and classifies but is forbidden to write, and prints the plan it would execute. Recommended first pass on any base worth keeping.
+- **Automatic backup before `audit-kb` applies.** `.axme-code/` is gitignored by design (D-026), so a pass that rewrites every file in the base had no safety net at all. A tarball is written to `.axme-code-backups/` first, and the audit aborts if it cannot be created. The undo command is printed with the results.
+- **Storage self-repair at session start.** `axme_context` runs the mechanical half of kb-doctor on every call and reports what it fixed. Waiting for a user to run a repair command is too late for the empty-slug defect: by then the second write has already destroyed the first.
+- **Knowledge-base hygiene reporting in `axme_context`.** Past a configurable threshold, one block reports the entry count with the compaction command, and separately reports how many entries overrun the catalog budget — two different problems with two different fixes.
+- **`axme_merge_memories`** — fold duplicates into one. The agent composes the merged text (which detail from each loser is worth keeping is judgment); the tool rewrites the survivor and archives the rest. Refuses outright if any source slug is missing, since a partial merge leaves the survivor claiming content that was never folded in.
+- **Regression coverage**: 50 new tests plus a `self-test` check that round-trips two non-Latin titles and asserts both are readable back under distinct filenames.
+
+### Changed
+
+- **The `CLAUDE.md` template's "During Work" section no longer says "save every successful approach".** That line, with no counterpart saying what *not* to save, is the root cause of the bloat this release addresses: a month of it produced 110 research diaries and 18 session handoffs in one base's memory. The template, the Cursor rules file, the MCP server instructions and the `axme_save_memory` description now all carry the same selection test — *would this help an agent a month from now who was not part of this investigation?* — with an explicit **negative** list alongside the positive one. The negative list is the operative half.
+- **The two-level format is now stated as a contract rather than implied.** `description` (memory) and `decision` (decision) are loaded into every future session; `body` and `reasoning` render as `## Details` / `## Reasoning`, are not loaded, and are returned in full by `axme_get_memory` / `axme_get_decision`. Measured usage before this change: `## Details` was non-empty in 11 memories out of 126 — the mechanism was right and simply never explained, so 91% of memories put everything in the paid layer. The tool descriptions, the schema field descriptions, the CLAUDE.md template and the server instructions now state which layer is which and what each costs. Also stated: do **not** split entries to meet the length — per-entry overhead multiplies by count. Cut down into the deferred layer, not across into more records.
+- **Save tools now return advisory notes instead of accepting anything silently.** An overlong `description` gets the concrete numbers ("1180 chars; the catalog renders 200; the last 980 will not be visible") and where the tail belongs. Near-duplicate titles are reported as merge candidates before a second half-record is created. `axme_save_decision` now says when title-dedup returned an existing decision unchanged, which previously read as a successful write. These are notes, never rejections: the write always lands, because a refused save loses the payload the agent just composed.
+- **The search-mode catalog marks truncated entries.** A cut line ends in `…[TRUNCATED]` and a header states how many of the entries are affected. Previously a truncated line was indistinguishable from a complete one, so an agent could not tell which entries it actually understood — and in practice fetched neither. The absence of the marker is now a guarantee that the entry is complete as shown, which is what makes writing to the budget worth doing: at that point search mode and full mode carry the same content.
+- **`audit-kb`'s prompt is the full compaction procedure** — classify into keep / compact / merge / archive, with the explicit negative list, "when in doubt, keep", "do not touch entries modified in the last 2 hours" (another session may be writing), and "do not rewrite history: keep both a retraction and what it retracts". It reindexes after applying, since compaction rewrites the text the embeddings index was built from and archival removes entries it still points at.
+- **The automatic write paths now carry the same contract as the manual ones.** This was the largest remaining gap: the guidance had been fixed in the CLAUDE.md template and the MCP tool descriptions, while the three paths that write *most* entries were untouched. The session auditor's JSON schema did not even include `body`, so the deferred layer was unreachable from the automatic path; the memory extractor's prompt said `body: keep short or omit — description must carry all meaning`, the exact inversion of the contract; and the `axme_begin_close` checklist had no negative list and no format rule. All three now state the selection test, the two-level format quoting the project's real `catalog.excerpt_chars`, and (for decisions) the prohibition on meta-decisions. The shared text lives in `src/storage/kb-format.ts` so the five surfaces cannot drift apart again.
+- **Meta-decisions are refused at the code level, not just discouraged.** A title that is nothing but a pointer between two ids ("D-020 absorbed by D-036") is rejected with a message naming `axme_archive_decision(superseded_by:)` as the operation that actually makes the edit. Matched on the title only — a decision that merely cites another in its body is legitimate and still saves.
+- **The KB-audit counter is surfaced where someone reads it.** It was incremented after every session audit and its recommendation written to the stderr of a *detached background worker*, i.e. nowhere. It now appears in `axme_context` alongside the size and format warnings.
+- `axme_save_decision` now explicitly instructs against meta-decisions ("D-020 absorbed by D-036"). Those describe edits to other decisions rather than decisions; `axme_archive_decision`'s `superseded_by` argument makes the edit instead. Nine such records were found in one base.
+
 ## [0.6.3] - 2026-06-25
 
 Reliability release for the `axme_save_memory` / `axme_save_decision` write tools, driven by two independent agent sessions that hit the same failure mode in production.

README.md

@@ -162,6 +162,22 @@ Your agent starts every session with full context: stack, decisions, patterns, g
 | **Handoff** | Where work stopped, blockers, next steps | "PR #17 open, waiting on review. Next: fix flaky test." |
 | **Worklog** | Session history and events | Timeline of all sessions and what was done |
 
+### Knowledge Base Hygiene
+
+A knowledge base that only grows becomes a tax on every session. Two levers keep it honest.
+
+**Write to the loaded layer, defer the rest.** Every memory and decision has a layer loaded into *every* future session (`description` / `decision`) and one that is not (`body` / `reasoning`, rendered as `## Details` / `## Reasoning`, returned in full by `axme_get_memory` / `axme_get_decision`). Keep the loaded layer to the rule plus one concrete fact, within `catalog.excerpt_chars` (default 200), and put measurements, paths, thresholds and line numbers below. Nothing is lost — it just stops being paid for by the sessions that never needed it. An entry within budget renders **complete** in the search-mode catalog, which is what makes search mode and full mode carry the same content.
+
+**Clean up with tools, not by hand.**
+
+| Command | Cost | What it does |
+|---|---|---|
+| `axme-code kb-doctor .` | free, instant | Finds broken slugs, leaked tool markup, entries over the catalog budget, duplicate titles. `--fix` repairs the mechanical ones. Exits 1 on findings — usable as a CI gate. |
+| `axme-code audit-kb . --dry-run` | one LLM run | Previews a compaction pass: what would be compacted, merged, archived. |
+| `axme-code audit-kb .` | one LLM run | Applies it. Takes a backup first, reindexes after, and reports the **measured** before/after — a pass that wrote nothing says so and exits non-zero. |
+
+Agents retire entries with `axme_archive_memory` / `axme_archive_decision`: files move to `.axme-code/archive/` with the reason stamped in, decisions are marked `superseded`/`revoked` before the move, and nothing is ever deleted.
+
 ### Safety Guardrails (100% Reliable)
 Hooks intercept tool calls **before execution** — not prompts. Even if the agent hallucinates a reason to run `rm -rf /`, the hook blocks it. This is hard enforcement at the Claude Code harness level, not a suggestion in a system prompt.
 
@@ -373,7 +389,8 @@ axme-code setup [path]       # Initialize project/workspace with LLM scan
 axme-code serve              # Start MCP server (called by Claude Code automatically)
 axme-code status [path]      # Show project status
 axme-code stats [path]       # Worklog statistics (sessions, costs, safety blocks)
-axme-code audit-kb [path]    # KB audit: dedup, conflicts, compaction
+axme-code kb-doctor [path]   # KB defect scan (no LLM, instant); --fix repairs mechanical ones
+axme-code audit-kb [path]    # KB compaction: compact, merge, archive (LLM); --dry-run to preview
 axme-code hook pre-tool-use  # PreToolUse hook handler (called by Claude Code)
 axme-code hook post-tool-use # PostToolUse hook handler
 axme-code hook session-end   # SessionEnd hook handler

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@axme/code",
-  "version": "0.6.3",
+  "version": "0.6.4",
   "description": "Persistent memory, decisions, and safety guardrails for Claude Code",
   "type": "module",
   "main": "./dist/server.js",