Skip to content

fix(deps): align eval with interface 0.39 - #488

Merged
drewstone merged 1 commit into
mainfrom
fix/interface-039-cohort
Jul 29, 2026
Merged

fix(deps): align eval with interface 0.39#488
drewstone merged 1 commit into
mainfrom
fix/interface-039-cohort

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

Why

agent-eval executed Interface 0.37 internally, so complete AgentProfileDiff experiments were rejected even when Runtime installed Interface 0.39. This aligns the exact Core/Interface cohort and releases the already-breaking unreleased supervisor-run changes as 0.136.0.

Proof

  • pnpm lint
  • pnpm typecheck
  • pnpm test — 4,227 passed, 3 skipped
  • pnpm build
  • pnpm run verify:package
  • uv lock --check --directory clients/python
  • New contract test seals model, harness, tools, MCP, hooks, and metadata through the existing AgentProfileDiff

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — aa171eae

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-07-29T22:27:57Z

@drewstone
drewstone merged commit 459519d into main Jul 29, 2026
2 checks passed

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Value Audit — sound

Verdict sound
Concerns 0 (none)
Heuristic 0.0s
Duplication 0.0s
Interrogation 73.0s (2 bridge agents)
Total 73.0s

💰 Value — sound

Lockstep cohort bump to interface 0.39/core 0.4.28 (release 0.136.0) with a new contract test proving the full AgentProfileDiff shape now seals — coherent, minimal, in-grain.

  • What it does: Bumps the exact-pinned dependency cohort in package.json (agent-core 0.4.26→0.4.28, agent-interface 0.37.0→0.39.0), mirrors it in scripts/verify-package-exports.mjs expectedDependencyCohort, releases as 0.136.0 across npm + clients/python (pyproject/uv.lock/init.py all 0.136.0), refreshes pnpm-lock.yaml, and adds a contract test that drives a complete AgentProfileDiff (model, harness, tools, m
  • Goals it achieves: Align agent-eval's INTERNAL Interface cohort with the one Runtime actually installs (0.39). Before this, agent-eval pinned 0.37 internally, so the broader AgentProfileDiff surface (model/harness/tools/mcp/hooks/metadata, added in 0.39) was rejected at the sealer even though the installed contract accepted it. The new test locks that the full diff shape round-trips, and the 0.135.4→0.136.0 minor bu
  • Assessment: This is exactly the intended use of the cohort-verification infrastructure introduced two commits ago in f6ce00d: bump the two exact-pinned deps in lockstep, update expectedDependencyCohort to match, refresh locks, extend a contract test. The experiment() refactor is clean — it parameterizes change with the old value as default, so existing tests are byte-for-byte unchanged and the new full-diff c
  • Better / existing approach: none — this is the right approach. The exact-pin + cohort-gate pattern (verify-package-exports.mjs:47-57, 684-731) IS the established mechanism for these bumps, introduced in f6ce00d and reused here verbatim. Grepped src for AgentProfileDiff — it is imported solely from @tangle-network/agent-interface (the substrate contract, correct layering per CLAUDE.md), with no local redefinition to consolida
  • Model: opencode/zai-coding-plan/glm-5.2
  • Bridge attempts: 2
  • Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content

🎯 Usefulness — sound

A minimal, correctly-scoped dependency cohort bump (Interface 0.37→0.39, Core 0.4.26→0.4.28) that unblocks complete AgentProfileDiff experiments through the real schema-validation gate, sealed by a new contract test and CI-enforced cohort verification.

  • Integration: Fully integrated. The load-bearing path is sealAgentProfileImprovementExperiment (src/contract/profile-measured-comparison.ts:114) → verifyAgentProfileImprovementExperiment → agentProfileImprovementExperimentSchema.parse (:136), a schema imported from @tangle-network/agent-interface. That schema is the single gate every profile-improvement experiment passes through; it is re-exported from the publ
  • Fit with existing patterns: Follows the established, CI-enforced cohort-alignment grain. The prior commit f6ce00d did the identical operation for Interface 0.37; this is the 0.39 repeat. scripts/verify-package-exports.mjs:47 pins exact versions and verifyPackedDependencyCohort (:684) proves a packed consumer installs one coherent copy of both agent-core and agent-interface, so the eval/Runtime skew the PR body describes cann
  • Real-world viability: Holds up off the happy path. The sealing path performs real zod .parse() validation (not a mock), so the test would throw if 0.39 still rejected any field. The complete-diff fixture covers every profile knob Interface defines (model+reasoningEffort, harness, tools, mcp servers, stop hooks with blocking, metadata), so an optimizer emitting any subset composes cleanly. The cohort check runs against
  • Model: opencode/zai-coding-plan/glm-5.2
  • Bridge attempts: 1

No concerns — sound change, no better or existing approach found. ✅


What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass What it asks
Heuristic Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication Do added function/class names already exist elsewhere in the repo?
Value Audit What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

value-audit · 20260729T222935Z

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants