Skip to content

fix(pr-management): reject empty dependency intersections - #1060

Merged
potiuk merged 1 commit into
apache:mainfrom
AlejandroMorgante:fix/dependency-empty-intersection
Aug 17, 2026
Merged

fix(pr-management): reject empty dependency intersections#1060
potiuk merged 1 commit into
apache:mainfrom
AlejandroMorgante:fix/dependency-empty-intersection

Conversation

@AlejandroMorgante

Copy link
Copy Markdown
Contributor

Summary

  • Classify an empty mandatory-constraint intersection as broken because the dependency graph is uninstallable, without requiring a concrete failing version.
  • Add a sixth dependency-compatibility regression and require every positive eval result to include the complete structured finding shape.
  • Grade each human-facing suggestion against the adopter-policy remediation expected by its fixture, so a contradictory fix cannot pass.

This is a focused follow-up to PR #1048, which established the dependency-evidence contract.

Type of change

  • Skill change (.claude/skills/<name>/) — eval fixtures updated below
  • Tool / bridge contract (tools/<system>/*.md)
  • Python package (tools/*/ with pyproject.toml)
  • Groovy reference impl
  • Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
  • Documentation (docs/, README.md, CONTRIBUTING.md)
  • Project template (projects/_template/)
  • CI / dev loop (prek, workflows, validators)
  • Other:

Test plan

  • prek run --all-files passes
  • For Python packages touched: uv run pytest / ruff check / mypy passes
  • For Groovy bridges touched: command-line invocation tested end-to-end
  • For skill changes: eval suite passes for the affected skill
    (uv run --project tools/skill-evals skill-eval tools/skill-evals/evals/pr-management-code-review/ rendered all 116 cases successfully)
  • For skill behaviour changes: a new or updated eval fixture is included in this PR
    (the dependency-compatibility suite passes 6/6 with Codex generating and Claude grading)
  • Other: a contradictory direct-bound suggestion fails when the expected remediation is a release marker; an output missing required finding fields fails deterministically

RFC-AI-0004 compliance

  • HITL — any new mutation is gated on explicit user confirmation
  • Sandbox — no new unrestricted host access; network reach declared in the adapter
  • Vendor neutrality — the skill continues to derive remediation from adopter policy, and fixtures use synthetic package names
  • Conversational + correctable — agentic-override path documented if behaviour is adopter-tunable
  • Write-access discipline — no autonomous outbound messages; drafts only, sent on confirmation
  • Privacy LLM — private content does not reach a non-approved LLM; redactor invoked where needed

Linked issues

Follow-up to PR #1048.

Notes for reviewers

The compatibility state space remains broken | compatible | unknown; an uninstallable graph is a broken graph, not a fourth state. The eval assertions and prose grading use the existing skill-evals mechanisms without changing the runner.

An empty constraint intersection has no concrete failing resolution, so the prior evidence rule could mistake an uninstallable graph for compatibility. Required output fields and case-specific suggestions also need enforcement so regressions cannot pass by omission.

Generated-by: Codex (GPT-5)
@AlejandroMorgante
AlejandroMorgante force-pushed the fix/dependency-empty-intersection branch from 37fdaa1 to 4c41ccc Compare August 2, 2026 00:25

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving — no findings. The reasoning holds and the gap was real, so it's
worth recording what I checked rather than just "LGTM".

The logic: with an empty effective intersection no resolution exists,
so the previous requirement of "one concrete supported resolution that
still fails" was impossible to satisfy. A definitively uninstallable graph
would therefore have been classified unknown (or the finding suppressed
as unsubstantiated), which is the wrong answer for the most clear-cut kind
of breakage there is. Classifying it broken is correct.

The part that is easy to get wrong and which this PR got right: the
surrounding prose is retro-scoped to "for a non-empty effective
intersection". Without that, the pre-existing rule — "if the available
evidence does not identify a concrete failing resolution, the runtime
incompatibility claim remains unsubstantiated and must not be raised"

would directly contradict the new empty-intersection rule. The state space
stays broken | compatible | unknown rather than growing a fourth member,
which is the right call.

What I verified:

  • The new has_required_finding_fields assertion actually behaves. I
    compiled the pattern and ran it: it matches a complete finding, rejects
    one missing file, and rejects []. That last point is why only the
    positive cases opt in via expected.json, and why case-3
    (has_empty_findings) correctly does not — the opt-in-by-key mechanism
    is used properly.
  • output-spec.md already declares every field the assertion demands
    (file, category, severity, runtime_compatibility,
    mandatory_paths_checked, metadata_coverage,
    supported_incompatible_resolution, reason, suggestion) with
    enumerations matching the regex alternatives, so it is not asserting a
    shape the prompt never asks for. It also omits recommended_action,
    which has_recommended_action already covers — no duplicated assertion.
  • case-6 is the discriminating case, not just another case.
    compat-core>=1.8.0,<2.0.0 against the query-base==2.0.0 transitive
    compat-core>=2.0.0 is genuinely empty, and pairing it with
    metadata_coverage: partial and
    supported_incompatible_resolution: false is exactly the combination the
    old rules would have mis-classified as unknown. Setting the flag
    false is also right: no resolution exists to be incompatible.
  • The per-fixture policies are consistent with their expected
    remediations.
    case-1 / case-2 supply a policy forbidding direct
    lower-bound changes → release_marker → "add the marker comment, do not
    change the bound"; case-5 / case-6 supply the opposite →
    direct_lower_bound → "raise the bound". Deliberately varying the policy
    per fixture is what makes the vendor-neutrality guarantee testable rather
    than asserted, and adding suggestion to grading-schema.json's
    prose_fields is what enforces it — a contradictory remediation now
    fails the rubric instead of going ungraded entirely.
  • Counts are updated in all three places that carry them: the spec
    (115 → 116), tools/skill-evals/README.md, and the suite README
    (step-4 5 → 6, with the description updated to name the uninstallable
    case).

The spec update in tools/spec-loop/specs/pr-management-family.md is
included in the same PR, which is what AGENTS.md's spec-sync rule asks
for — the specs do not fall behind the behaviour they describe.


This review was drafted by an AI-assisted tool and
confirmed by a Magpie maintainer. The maintainer
approving this PR has read the findings and signed off. If
something feels off, please reply on the PR and a maintainer
will follow up.

More on how Magpie handles maintainer review:
CONTRIBUTING.md.

@potiuk
potiuk merged commit bdae276 into apache:main Aug 17, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants