Wire post-swap monitor into live 2-policy serving (rollback was dead code) - #301
Merged
rylinjames merged 1 commit intoAug 12, 2026
Merged
Conversation
…owercase bug post_swap_monitor.py and rollback.py existed as real, unit-tested-in-isolation code but were never imported outside pro/__init__.py and their own test files -- no CLI path or live server code reached them, so an auto-rollback on production regression was effectively dead code. Adds a force_drain override to PolicyCrashTracker (kept distinct from crash-count-derived drains so it doesn't corrupt that telemetry), wires PostSwapMonitor + RollbackHandler into runtime/server.py's 2-policy /act path using T1 (safety-clamp) and T3 (safety-violation webhook count) signals already computed per-request, and adds --rollback-sensitivity / --post-swap-baseline-clamp-rate CLI flags. T2 (action cosine-similarity to the previous model) is intentionally left unwired live -- computing it per request would double inference cost by running both policies on every call -- and is documented as such rather than faked. NOTE: this PR does NOT include the docker-smoke.yml fix for the same root cause (raw docker pull against ghcr.io/github.repository is mixed-case and therefore invalid). That fix is ready but blocked on push: this session's git token lacks the `workflow` OAuth scope GitHub requires to push changes under .github/workflows/. Delivered out-of-band to the repo owner as a patch to apply directly. New tests: 6 in test_two_policy_dispatcher.py covering force_drain, clear_forced_drain, and priority-over-organic-drain; 3 in a new test_post_swap_rollback_integration.py proving PostSwapMonitor and RollbackHandler compose correctly against a REAL TwoPolicyDispatcher (the existing rollback tests only exercised RollbackHandler against a stub router matching an API PolicyRouter never actually implements). Full suite: 3192 passed, 100 skipped, 0 failed (was 3183/100/0 before this change). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
rylinjames
deleted the
fix/wire-post-swap-rollback-and-docker-smoke-lowercase
branch
August 12, 2026 17:28
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pro/post_swap_monitor.pyandpro/rollback.pywere real, well-designed, unit-tested-in-isolation code, but neither was imported anywhere outsidepro/__init__.pyand their own test files. No CLI path or live server code reached them. If a promoted policy regressed in production, nothing in the running system would notice or auto-rollback.PolicyCrashTracker.force_drain()/clear_forced_drain()as an explicit external-override mechanism, kept distinct from the existing crash-count-derived drain path so a monitor trip doesn't corrupt crash telemetry those counters also drive.PostSwapMonitor+RollbackHandlerintoruntime/server.py's 2-policy/actpath: T1 (safety-clamp) and T3 (safety-violation webhook count) use signals already computed per-request (result["guard_summary"]["clamp_count"],result["safety_violations"]); a trip callsdispatcher.force_drain()(same mechanism an organic crash-count drain already uses) andRollbackHandler.execute()for the audit trail.--rollback-sensitivityand--post-swap-baseline-clamp-ratetotether serve.PostSwapMonitoralready supportscos_to_previous_model=Nonefor exactly this case ("T2 not checked this episode"), so this is documented, not silently faked.workflowOAuth scope required to modify.github/workflows/*. Sending that fix to the repo owner directly as a patch to apply.Test plan
pytest tests/, 3192 passed, 100 skipped, 0 failed (baseline before this change: 3183/100/0, the delta is exactly the 9 new tests below, no regressions)test_two_policy_dispatcher.py:force_drainoverrides routing without touching crash counters, persists across multiple predicts,clear_forced_drainrestores normal routing, rejects unknown slots, and takes priority over an organic crash-derived draintest_post_swap_rollback_integration.py:PostSwapMonitor+RollbackHandlercomposed against a realTwoPolicyDispatcher(not the existing rollback tests'_StubRouter, which encodes arouter.set_active()APIPolicyRouternever actually implements), proves T2 and T3 trips actually drain the monitored slot on the next live request, and that sub-sensitivity-threshold trips correctly do not fire🤖 Generated with Claude Code