Skip to content

ci(warden): Reduce PR review cost and enforce timeout#480

Merged
cameroncooke merged 8 commits into
mainfrom
codex/trim-warden-pr-checks
Jul 21, 2026
Merged

ci(warden): Reduce PR review cost and enforce timeout#480
cameroncooke merged 8 commits into
mainfrom
codex/trim-warden-pr-checks

Conversation

@cameroncooke

Copy link
Copy Markdown
Collaborator

PR Warden currently runs both repository and centrally managed workflows. Across 49 recent events, the retained snapshot and bug-finding checks produced 34 of 37 logical findings, while the other automatic checks produced only three. The centrally managed run on #479 also remained active for 58 minutes.

This removes the duplicate repository workflow and narrows automatic PR review to the snapshot-fixture and find-bugs skills with lower turn and finding limits. Runtime-boundary, test-boundary, and tool-contract reviews remain available manually, with concise agent instructions to run them for matching high-risk changes. Scheduled and local security reviews remain outside the PR critical path.

A default-branch watchdog now monitors each PR Warden run by exact run ID and requests cancellation at ten minutes, including queue time. Its focused tests cover the deadline and completion-versus-cancellation races.

Remove the duplicate repository Warden lane and retain only the two productive automatic PR skills.

Add a ten-minute watchdog and manual guidance for targeted domain reviews.
Copilot AI review requested due to automatic review settings July 21, 2026 12:23
@pkg-pr-new

pkg-pr-new Bot commented Jul 21, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/xcodebuildmcp@480

commit: 062afdb

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces automatic PR-time Warden review cost by narrowing which skills run on pull_request events, and adds a repository-side watchdog workflow to cancel long-running Warden runs (including queue time) to avoid excessively long checks.

Changes:

  • Remove the repository-owned PR Warden workflow and narrow PR-triggered Warden skills to snapshot-fixture review + find-bugs with stricter turn/finding limits.
  • Add a “Warden watchdog” workflow plus a Node-based watchdog script to cancel Warden runs that exceed a fixed runtime budget.
  • Update agent guidance docs and the snapshot-fixture skill guardrails to reflect the new intended review workflow and limits.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
warden.toml Shrinks PR-triggered skills to a small high-signal set and caps turns/findings; keeps broader reviews scheduled/local.
scripts/warden-watchdog.mjs Implements polling + cancellation logic for a specific workflow run ID.
scripts/__tests__/warden-watchdog.test.mjs Adds focused watchdog behavior tests (currently not wired into the repo’s Vitest CI run).
CLAUDE.md Adds guidance to manually run high-risk Warden reviews before handoff.
AGENTS.md Mirrors the manual Warden review guidance for agents.
.github/workflows/warden.yml Removes the repository PR-triggered Warden workflow (expects org-managed workflow to be the PR path).
.github/workflows/warden-watchdog.yml Adds a workflow_run-triggered watchdog job intended to cancel long Warden runs.
.agents/skills/xcodebuildmcp-snapshot-fixture-review/SKILL.md Tightens review guardrails/limits and explicitly discourages running expensive suites during automated PR review.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/warden-watchdog.yml Outdated
Comment thread .github/workflows/warden-watchdog.yml
Comment thread scripts/warden-watchdog.mjs Outdated
Comment thread scripts/__tests__/warden-watchdog.test.mjs Outdated
@cameroncooke
cameroncooke marked this pull request as ready for review July 21, 2026 12:35
Comment thread warden.toml
Comment thread scripts/__tests__/warden-watchdog.test.mjs
Start monitoring requested runs so queue time is enforced, pin Node 24, make deadline polling deterministic, and run watchdog tests through npm test.
Copilot AI review requested due to automatic review settings July 21, 2026 12:38
Keep runtime, test, and tool contract skills configured with local-only triggers so the agent handoff commands remain usable without adding PR CI cost.
Comment thread scripts/warden-watchdog.mjs

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Comment thread package.json Outdated
Comment thread scripts/warden-watchdog.mjs
Copilot AI review requested due to automatic review settings July 21, 2026 12:46
The workflow_run trigger already selects Warden and provides the exact run ID. Validate only the pull request event so org-required runs are still monitored when GitHub omits their workflow name.
Run the watchdog suite through npm posttest so targeted Vitest arguments are not forwarded to the Node test runner.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

package.json:50

  • "test" now chains vitest run && npm run test:warden-watchdog, which breaks the common npm test -- <pattern-or-file> workflow: the extra args will be appended to the chained command and can end up being forwarded to the watchdog Node test runner (e.g., existing skill docs reference npm test -- src/runtime/__tests__/tool-invoker.test.ts). That can cause the watchdog test step to try to execute non-Node test files and fail unexpectedly.

Consider keeping npm test as the Vitest entrypoint (so npm test -- ... remains predictable), and introducing a separate CI/aggregate script (e.g. test:ci) that runs both suites; then update CI/docs/skills to use that aggregate script.

    "test": "vitest run",
    "posttest": "npm run test:warden-watchdog",

Copilot AI review requested due to automatic review settings July 21, 2026 12:52

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Comment thread scripts/warden-watchdog.mjs Outdated
Comment thread scripts/__tests__/warden-watchdog.test.mjs Outdated
Recheck terminal state after cancellation, monitor rerun attempts from their current start time, and keep watchdog code in standard quality checks. Add concise review-readiness guidance and align the manual test-boundary review scope.
Copilot AI review requested due to automatic review settings July 21, 2026 16:13

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 3af57f2. Configure here.

Comment thread scripts/warden-watchdog.mjs
Comment thread scripts/warden-watchdog.mjs
Copilot AI review requested due to automatic review settings July 21, 2026 16:26
Comment thread scripts/warden-watchdog.mjs

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

scripts/warden-watchdog.mjs:74

  • monitorWardenRun returns { cancelled: true } as soon as the cancel request is accepted even when the run is still not completed (the common case right after hitting the cancel endpoint). This makes main() throw Cancelled Warden run… without confirming the run actually reached conclusion: 'cancelled', and the watchdog stops monitoring while the run may continue executing.
  const cancellationAccepted = await cancelRun();
  run = await getRun();
  if (run.status === 'completed') {
    return { cancelled: cancellationAccepted && run.conclusion === 'cancelled', ignored: false };
  }

  if (cancellationAccepted) {
    return { cancelled: true, ignored: false };
  }

Copilot AI review requested due to automatic review settings July 21, 2026 16:33

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

scripts/warden-watchdog.mjs:74

  • After a successful cancelRun() request, monitorWardenRun returns { cancelled: true } immediately when the attempt-scoped run is still active (lines 72-74). GitHub can accept a cancellation request while the run status is still propagating, and in some races the run may actually complete successfully; this would make the watchdog throw even though nothing was cancelled.

Consider double-checking the generic run endpoint (getCurrentRun) before deciding the run was cancelled/still-active, so propagation lag doesn’t produce a false timeout failure.

  if (cancellationAccepted) {
    return { cancelled: true, ignored: false };
  }

@cameroncooke
cameroncooke merged commit f73e3a5 into main Jul 21, 2026
22 checks passed
@cameroncooke
cameroncooke deleted the codex/trim-warden-pr-checks branch July 21, 2026 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants