fix(deps): align eval with interface 0.39 - #488
Conversation
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — aa171eae
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-29T22:27:57Z
tangletools
left a comment
There was a problem hiding this comment.
🟢 Value Audit — sound
| Verdict | sound |
| Concerns | 0 (none) |
| Heuristic | 0.0s |
| Duplication | 0.0s |
| Interrogation | 73.0s (2 bridge agents) |
| Total | 73.0s |
💰 Value — sound
Lockstep cohort bump to interface 0.39/core 0.4.28 (release 0.136.0) with a new contract test proving the full AgentProfileDiff shape now seals — coherent, minimal, in-grain.
- What it does: Bumps the exact-pinned dependency cohort in package.json (agent-core 0.4.26→0.4.28, agent-interface 0.37.0→0.39.0), mirrors it in scripts/verify-package-exports.mjs expectedDependencyCohort, releases as 0.136.0 across npm + clients/python (pyproject/uv.lock/init.py all 0.136.0), refreshes pnpm-lock.yaml, and adds a contract test that drives a complete AgentProfileDiff (model, harness, tools, m
- Goals it achieves: Align agent-eval's INTERNAL Interface cohort with the one Runtime actually installs (0.39). Before this, agent-eval pinned 0.37 internally, so the broader AgentProfileDiff surface (model/harness/tools/mcp/hooks/metadata, added in 0.39) was rejected at the sealer even though the installed contract accepted it. The new test locks that the full diff shape round-trips, and the 0.135.4→0.136.0 minor bu
- Assessment: This is exactly the intended use of the cohort-verification infrastructure introduced two commits ago in f6ce00d: bump the two exact-pinned deps in lockstep, update expectedDependencyCohort to match, refresh locks, extend a contract test. The experiment() refactor is clean — it parameterizes change with the old value as default, so existing tests are byte-for-byte unchanged and the new full-diff c
- Better / existing approach: none — this is the right approach. The exact-pin + cohort-gate pattern (verify-package-exports.mjs:47-57, 684-731) IS the established mechanism for these bumps, introduced in f6ce00d and reused here verbatim. Grepped src for AgentProfileDiff — it is imported solely from @tangle-network/agent-interface (the substrate contract, correct layering per CLAUDE.md), with no local redefinition to consolida
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 2
- Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content
🎯 Usefulness — sound
A minimal, correctly-scoped dependency cohort bump (Interface 0.37→0.39, Core 0.4.26→0.4.28) that unblocks complete AgentProfileDiff experiments through the real schema-validation gate, sealed by a new contract test and CI-enforced cohort verification.
- Integration: Fully integrated. The load-bearing path is sealAgentProfileImprovementExperiment (src/contract/profile-measured-comparison.ts:114) → verifyAgentProfileImprovementExperiment → agentProfileImprovementExperimentSchema.parse (:136), a schema imported from @tangle-network/agent-interface. That schema is the single gate every profile-improvement experiment passes through; it is re-exported from the publ
- Fit with existing patterns: Follows the established, CI-enforced cohort-alignment grain. The prior commit f6ce00d did the identical operation for Interface 0.37; this is the 0.39 repeat. scripts/verify-package-exports.mjs:47 pins exact versions and verifyPackedDependencyCohort (:684) proves a packed consumer installs one coherent copy of both agent-core and agent-interface, so the eval/Runtime skew the PR body describes cann
- Real-world viability: Holds up off the happy path. The sealing path performs real zod .parse() validation (not a mock), so the test would throw if 0.39 still rejected any field. The complete-diff fixture covers every profile knob Interface defines (model+reasoningEffort, harness, tools, mcp servers, stop hooks with blocking, metadata), so an optimizer emitting any subset composes cleanly. The cohort check runs against
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
No concerns — sound change, no better or existing approach found. ✅
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
Why
agent-evalexecuted Interface 0.37 internally, so completeAgentProfileDiffexperiments were rejected even when Runtime installed Interface 0.39. This aligns the exact Core/Interface cohort and releases the already-breaking unreleased supervisor-run changes as 0.136.0.Proof
pnpm lintpnpm typecheckpnpm test— 4,227 passed, 3 skippedpnpm buildpnpm run verify:packageuv lock --check --directory clients/pythonAgentProfileDiff