Skip to content

feat(runtime): establish managed mutation lifecycle authority - #3741

Merged
zhiiw merged 8 commits into
apache:mainfrom
zhiiw:codex/managed-mutation-lifecycle-authority
Aug 30, 2026
Merged

feat(runtime): establish managed mutation lifecycle authority#3741
zhiiw merged 8 commits into
apache:mainfrom
zhiiw:codex/managed-mutation-lifecycle-authority

Conversation

@zhiiw

@zhiiw zhiiw commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Primary invariant

From a managed Write/Edit T1 until a trusted terminal state, one workspace instance has exactly one durable mutation owner. The T1 path and fixed execution profile are derived from the durable tool call; terminal publication is restricted to an owner-verified successor, a committed no-change result, or a committed no-effect failure. Managed execution never falls back to generic T2.

Ownership and atomic boundaries

  • Immutable RuntimeEvents remain the sole durable truth; SQLite reservations and workspace projections are rebuildable projections.
  • T1 binds one SHA-1 base, one exact path under Gitoxide path-policy v3, and one fixed managed execution profile.
  • The repository owner supplies an opaque candidate capability; raw Git OIDs/descriptors cannot advance the head.
  • Runtime invokes only a dedicated pure transform in managed mode and owns one bounded, strict-JSON immutable result snapshot.
  • Successor acceptance atomically commits T2, successor fact, projections, and canonical head.
  • No-change success and proven no-effect failure each have a reserved atomic terminal writer that commits T2 and releases the reservation without advancing the head.
  • Generic append/T2 APIs cannot persist workspace authority or managed terminal facts.

Failure and rollback

  • Failure before or inside T1 rolls back the entire admission.
  • Failure after T1 without owner proof remains unsettled and fail-stops.
  • A crash inside successor or no-effect terminal commit rolls back the whole transaction.
  • A crash after commit converges by exact immutable retry; the operation is not re-executed.
  • Projections rebuild from RuntimeEvents; malformed or conflicting authority fails closed.

Scope

This PR establishes the persistence and Runtime lifecycle authority. It deliberately does not implement Git candidate capture, filesystem mutation, Runtime Host composition, or Desktop/CLI enablement.

Because this Draft has no production Write/Edit consumer, it keeps the released SQLite schema at version 14 and does not pre-commit a speculative online-query index. The current immutable full-ledger validation remains correct but is not a production-scale hot-path claim. A bounded RuntimeEvent index/query must be introduced with the real consumer, once its access shape is known, without making a rebuildable projection the source of truth.

The branch was rebuilt on current upstream/main. It spans schema-owned tables already introduced by this slice and Runtime protocol, but not Host lifecycle or platform filesystem I/O.

Verification

  • @maka/core, @maka/storage, and @maka/runtime builds pass.
  • Focused authority/runtime/recovery verification passes, including real child-process crash/reopen and real multiprocess reservation/migration contention.
  • After removing the speculative schema-15 index, the Storage migration/crash/concurrency set reports 125 passed, 0 failed, 1 existing POSIX-only conditional skip.
  • git diff --check passes.
中文说明

主要不变量

从 managed Write/Edit 的 T1 到可信终态,同一个 workspace instance 只能有一个 durable mutation owner。T1 的路径和固定 execution profile 来自 durable tool call;终态只能是 owner 验证的 successor、已提交的 no-change success,或已证明无副作用的失败。managed execution 禁止回退 generic T2。

Owner 与原子边界

  • immutable RuntimeEvents 是唯一 durable truth;SQLite reservation 与 workspace projection 都只是可重建投影。
  • T1 绑定一个 SHA-1 base、一个符合 Gitoxide path-policy v3 的精确路径,以及一个固定 managed execution profile。
  • repository owner 只签发 opaque candidate capability;裸 Git OID/descriptor 无权推进 head。
  • managed mode 只执行专用 pure transform;Runtime 独占一次有界 strict-JSON traversal 产生的不可变结果快照。
  • successor acceptance 在一个事务中提交 T2、successor fact、projection 和 canonical head。
  • no-change success 与已证明 no-effect failure 各自由 reserved atomic terminal writer 提交 T2、释放 reservation,但不推进 head。
  • generic append/T2 API 无权写 workspace authority 或 managed terminal fact。

范围与 schema 决策

本 PR 只建立 persistence 与 Runtime lifecycle authority,不包含 Git candidate capture、文件 mutation、Runtime Host composition 或 Desktop/CLI enablement。

当前 Draft 没有生产 Write/Edit consumer,因此 SQLite schema 保持版本 14,不提前增加 speculative schema 15/index。现有 immutable full-ledger 校验在正确性上成立,但不声称已经是生产规模热路径。等真实 consumer 确定访问形状后,再引入不依赖可重建 projection 的有界 RuntimeEvent 索引读取。

三个 workspace build 通过;撤回 schema 15 后,Storage migration/crash/concurrency 套件为 125 passed、0 failed、1 个既有 POSIX 条件跳过;git diff --check 通过。

@zhiiw
zhiiw marked this pull request as draft August 25, 2026 02:01
@zhiiw

zhiiw commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

The required CI check is now green on cf63fb9.

The first run exposed two issues, both fixed at their actual owner boundary:

  • Runtime terminal classification: the refactor retained only the durable isError bit and used it for live status, which collapsed subagent aborted into error. Live status now continues to use the full canonical terminal classifier, while the durable envelope keeps its boolean error representation.
  • Legacy schema fixture rewind: the schema-4 tests synthesize an old database by rewinding a current one. They did not remove the new schema-14 reservation table, so migration attempted to create an artifact a real schema-4 database cannot contain. The rewind now removes that table before setting user_version=4.

Verification after the fix:

  • The four original failing tests pass.
  • The related recovery and managed-mutation suites pass 122/122.
  • The full required CI check passes in 7m49s.
中文说明

required CI 已在 cf63fb9 全绿。

首轮 CI 暴露了两个问题,并且都在真正的 owner 边界修复:

  • Runtime terminal classification:重构后只保留 durable isError 位并误用于 live status,导致 subagent 的 aborted 被压成 error。现在 live status 继续使用完整 canonical classifier,durable envelope 仍保留布尔 error 表示。
  • legacy schema fixture rewind:schema-4 测试通过把当前数据库倒带来构造旧库,但漏删 schema-14 新增的 reservation table,迁移因此重复建表;真实 schema-4 库不会存在这张表。现在在设置 user_version=4 前会删除它。

修复后:

  • 原始 4 条失败测试全部通过;
  • 相关 recovery/managed-mutation 套件 122/122 通过;
  • required CI 全量通过,用时 7m49s。

@zhiiw
zhiiw force-pushed the codex/managed-mutation-lifecycle-authority branch from cf63fb9 to f5cac84 Compare August 26, 2026 09:26
@zhiiw

zhiiw commented Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review. I rebuilt the branch on current upstream/main and addressed the authority gaps at the owner boundaries. The current head is 2f0601648.

Resolved items:

  1. T1 path/profile authority

    • managed_mutation_v2 is SHA-1 only and binds exactly one expectedPath, path-policy v3, and one fixed managed execution-profile digest.
    • Runtime and SQLite both compare the durable function_call.args.path with that path before T1.
    • The prior multi-path/Delete/SHA-256 generalization was removed.
  2. Real no-effect terminal authority

    • A reserved managed_mutation_terminal_v1 fact and atomic SQLite writer now commit either no-change success or proven no-effect failure.
    • The transaction commits T2 and releases the exact reservation without advancing the head; generic writers cannot use this lane.
    • Real child-process tests cover death inside the transaction and after commit.
  3. Opaque successor admission

    • Successor commit no longer accepts raw Git OIDs/descriptors. It requires an opaque candidate object and a process-local repository-owner verifier before the private SQLite writer receives verified data.
  4. Pure managed execution path

    • Managed mode requires and invokes managedMutationTransform; it never calls the ordinary mutable tool.impl path.
  5. Fixed result-resource profile

    • Managed snapshotting has non-overridable caps: 1 MiB, depth 64, and 65,536 nodes/properties/array entries. Caller limits can only reduce the byte budget.
  6. Online scan finding: explicitly deferred without a new durable contract

    • I initially introduced schema 15 with a RuntimeEvent expression index, then removed it after scope review.
    • This Draft has no production Write/Edit consumer, so committing a permanent migration for a not-yet-established access shape was disproportionate.
    • Schema remains 14. The current immutable full-ledger validation remains the correctness path and does not use a rebuildable projection as truth. A bounded immutable-event query/index is now an explicit consumer-time gate, not a claim of this inert slice.

Verification after removing the speculative migration:

  • Storage build passes.
  • Storage authority/migration/crash/concurrency set: 125 passed, 0 failed, 1 existing POSIX-only conditional skip.
  • The earlier Core/Runtime focused authority tests remain unchanged by this storage-only scope reduction.
  • git diff --check passes.
中文说明

当前 head 为 2f0601648。T1 单路径与固定 profile、no-effect 原子终态、opaque candidate、pure transform 和固定结果资源上限均已在各自 owner 边界完成。

关于在线全量扫描:我最初加了 schema 15 表达式索引,但复核范围后已撤回。这个 Draft 还没有生产 Write/Edit consumer,在访问形状尚未确定时提前固化永久 migration/index,成本高于实际收益。现在 schema 保持 14;当前实现继续从 immutable RuntimeEvents 校验事实,不会借助可重建 projection 决定事实。等真实 consumer 接入时,再把有界 immutable-event query/index 作为明确的生产门槛实现。

撤回 schema 15 后,Storage build 通过;migration/crash/concurrency 定向套件 125 passed、0 failed、1 个既有 POSIX 条件跳过;git diff --check 通过。

@M4n5ter
M4n5ter force-pushed the codex/managed-mutation-lifecycle-authority branch 3 times, most recently from 0746b8c to 464eb0d Compare August 26, 2026 09:54
@zhiiw
zhiiw marked this pull request as ready for review August 26, 2026 13:18
@zhiiw
zhiiw force-pushed the codex/managed-mutation-lifecycle-authority branch from 464eb0d to a5db232 Compare August 26, 2026 13:22
@zhiiw

zhiiw commented Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

CI has been refreshed on a5db23226 after rebuilding the seven PR commits on current main@e55fa3bb3.

The previous failure was isolated to the existing Runtime Host test does not take over a generation-mismatched Host before it is ready: its cleanup raced with a writer under the shared Runtime Host control namespace and Linux returned ENOTEMPTY. This PR has no diff in the failing test, Host kernel, or registration lifecycle; Windows recovery passed, and recent main runs passed the same suite. I therefore did not add unrelated Host cleanup behavior to the managed-mutation authority slice.

Verification after the rebuild:

  • Core, Storage, Runtime, and Runtime Host builds passed.
  • Core/Runtime focused suites: 126 passed.
  • Storage authority/crash/concurrency suites: 89 passed.
  • git diff --check passed.
  • The branch is 0 commits behind current main and GitHub CI is running again.
中文说明

已将 7 个 PR 提交重放到最新 main@e55fa3bb3,新 head 为 a5db23226,CI 已重新触发。

上次失败已定位为既有 Runtime Host 测试 does not take over a generation-mismatched Host before it is ready 的清理竞态:共享 control namespace 仍有写入时递归删除,Linux 返回 ENOTEMPTY。本 PR 对失败测试、Host kernel 和 registration 生命周期都没有 diff;Windows recovery 已通过,近期 main 也能通过同一套测试。因此没有把无关的 Host 清理修改塞进 mutation authority PR。

重建后本地验证:四个 workspace 构建通过,Core/Runtime 126 个定向测试通过,Storage authority/crash/concurrency 89 个测试通过,git diff --check 通过。当前分支不再落后 main,GitHub CI 正在重新运行。

@likun666661

Copy link
Copy Markdown
Member

I think the core problem of this PR is sound, and it is different from #3857:

The lifecycle here is:

  1. T1 freezes the base/path/profile and acquires the single durable mutation reservation.
  2. The mutation runs and may produce a candidate.
  3. T2 atomically records the tool result, records the accepted successor (if any), advances the SQLite canonical head, and releases the reservation.
  4. If the result is uncertain, the reservation remains parked for recovery instead of guessing.

From an Occams razor perspective, I do not think the T1/T2 state machine is accidental complexity. The reservation and the explicit terminal cases are the minimum needed to avoid two writers, partial settlement, and unsafe fallback behavior.

A few improvements I would make before wiring this to the staging engine:

  1. The execution profile digest is currently a hard-coded SHA-256 constant. It should either be derived from a canonical profile representation that actually describes the mutation semantics, or be named as a version/profile identifier. Otherwise the value can remain unchanged while behavior changes, so it does not really bind the execution contract.

  2. An exact retry may return the historical successor after the canonical head has advanced again. That behavior is reasonable, but calling the returned value head is misleading. A name such as acceptedHeadAtCommit or committedSuccessor would make it clear that callers must reread the current canonical head when they need current state.

  3. The successor path already uses an opaque candidate capability/verifier, but the no-change and failed-with-no-effect terminal paths currently accept a plain outcome. When this is integrated with feat(runtime-host): add Gitoxide candidate and accepted-tree read data plane #3857, those paths should consume equivalent evidence: a verified no-change capability or a proof that a failed attempt had no workspace effect. Otherwise the coordinator is trusting the caller for the exact claims that allow it to release the reservation safely.

  4. The full RuntimeEvent scan is fine as a correctness-first implementation, but it is unbounded. I agree with treating an indexed lookup/bounded replay path as a production gate before enabling real Write/Edit consumers.

So my conclusion is: keep the lifecycle model; simplify and strengthen the contracts at its boundaries. The clean end-to-end composition should be: this PR reserves the mutation at T1, #3857 creates/verifies staged B, and this PR settles the operation at T2. Physical Git ref promotion can remain a later, explicit owner.

@zhiiw
zhiiw force-pushed the codex/managed-mutation-lifecycle-authority branch from da66f6c to 082cd2f Compare August 29, 2026 14:36
@zhiiw

zhiiw commented Aug 29, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the focused follow-up. I rebuilt the seven PR commits on current main and addressed the three contract issues at their owner boundaries. The new head is 082cd2f.

  1. The execution-profile digest is no longer an opaque label. Core now owns a canonical v1 profile specification covering the pure Write/Edit transform, SHA-1 object format, path policy, strict-JSON result limits, terminal authority, and forbidden generic fallback. The digest is named explicitly as a digest, and a regression independently recomputes SHA-256 from the canonical representation. Runtime consumes the result limits directly from that same specification.

  2. WorkspaceSuccessorCommitResult now returns committedSuccessor instead of head. Exact retry still returns the historical immutable successor accepted by that operation, while callers that need current state must use the canonical head reader.

  3. No-change and failed-with-no-effect settlement now require an opaque owner-issued no-effect capability. A registered internal verifier restores the operation, dispatch, workspace-instance, and terminal-kind claim; SQLite compares all four fields with the durable terminal event before it can commit T2 and release the reservation. A plain outcome and a mismatched capability are both rejected while the reservation remains active.

  4. I kept the full immutable RuntimeEvent scan unchanged. This slice still has no production Write/Edit consumer, so its real online access pattern is not yet established. The architecture document continues to make a bounded immutable-event query/index an explicit production gate before that consumer is enabled; rebuild remains based on RuntimeEvents as the sole durable truth. This avoids introducing another speculative schema contract while preserving fail-closed correctness.

Verification on the rebuilt branch:

  • Core, Runtime, and Storage builds pass.
  • Core authority suites: 26/26.
  • Runtime durable-boundary suite: 42/42.
  • Storage authority, real-process crash, and multiprocess concurrency suites: 58/58.
  • Biome check and git diff --check pass.
  • The branch is 0 commits behind the main revision used for the rebuild.
中文说明

感谢这轮边界清晰的反馈。我已把 7 个 PR 提交重建到当前 main,并在各自 owner 边界完成前三项修复。新 head 为 082cd2f

  1. execution profile digest 不再是孤立的硬编码标签。Core 现在拥有 canonical v1 profile spec,明确绑定 pure Write/Edit transform、SHA-1 object format、path policy、strict-JSON 结果上限、terminal authority 和禁止 generic fallback。digest 也明确以 digest 命名;回归测试会从独立 canonical 表示重新计算 SHA-256。Runtime 的结果预算直接读取同一份 spec。

  2. WorkspaceSuccessorCommitResult 的 head 已改名为 committedSuccessor。exact retry 返回的是该 operation 当时接受的 immutable historical successor;需要当前状态的调用者必须重新读取 canonical head。

  3. no-change 与 failed-with-no-effect 现在必须携带 mutation owner 签发的 opaque no-effect capability。内部 verifier 还原 operation、dispatch、workspace instance 和 terminal kind,SQLite 在提交 T2、释放 reservation 以前逐字段绑定 durable terminal event。普通 outcome 或不匹配的 capability 都会被拒绝,reservation 保持 active。

  4. immutable RuntimeEvent 全量扫描暂未改动。当前切片仍没有生产 Write/Edit consumer,真实在线访问形状尚未确定;架构文档继续把 bounded immutable-event query/index 明确列为 consumer 启用前的 production gate,完整 rebuild 仍以 RuntimeEvents 为唯一 durable truth。这样不会为了尚未确定的查询形状提前固化新的 schema 合同。

验证结果:Core/Runtime/Storage build 通过;Core authority 26/26,Runtime durable-boundary 42/42,Storage authority + 真实进程 crash + 多进程 concurrency 58/58;Biome 与 git diff check 均通过。

@likun666661 likun666661 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. The T1/T2 managed-mutation lifecycle boundary is sound: this PR owns durable reservation and atomic settlement, while candidate construction remains outside this authority. The latest update closes the reviewed contract gaps by binding the execution-profile digest to a canonical specification, returning committedSuccessor rather than a misleading current head, and requiring owner-issued opaque no-effect evidence before releasing a reservation. The deferred bounded RuntimeEvent query remains explicitly documented as a production-consumer gate.

@zhiiw
zhiiw merged commit 8c491e6 into apache:main Aug 30, 2026
2 checks passed
saltand pushed a commit to saltand/maka-agent that referenced this pull request Aug 31, 2026
…#3741)

* feat(storage): establish workspace successor authority

* feat(storage): reserve durable managed mutations

* feat(runtime): enforce managed mutation settlement

* docs(runtime): define managed mutation lifecycle authority

* fix(runtime): preserve terminal status across managed settlement

* fix(runtime): close managed mutation authority gaps

* refactor(storage): defer managed mutation query indexing

* fix(runtime): tighten managed mutation evidence
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/XL Over 1000 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants