Improve duplicate-surfacing search recall (in:title + high-recall queries) - #6410
Merged
denelon merged 1 commit intoJul 29, 2026
Merged
Conversation
…ries) The Duplicate Surfacing agent judges semantically but retrieves via GitHub's full-text issue search, whose relevance ranking buries older, sparsely-worded issues. On issue microsoft#6405 (Support source type detection) the agent's jargon-heavy queries missed the true duplicate microsoft#1477 (The type of a source should be automatically detected when possible) — GitHub full-text search never returned it, even for its own title words. Update the 'How to search' guidance to: - Always run at least two in:title concept searches (2-3 core nouns). - Prefer short, high-recall queries; avoid piling exact CLI tokens into one query, which shrinks the result set. - Vary noun/verb vocabulary (weak stemming) and scan ~20-30 results per search. Validated via 'gh aw trial' replaying microsoft#6405 against the live corpus: the agent now surfaces microsoft#1477 with HIGH confidence. Reported by Trenly on microsoft#6406. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3701d64a-96ba-42f1-be59-b4e7d2a2820a
denelon
marked this pull request as ready for review
July 29, 2026 18:40
Trenly
approved these changes
Jul 29, 2026
JohnMcPMS
approved these changes
Jul 29, 2026
This was referenced Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📖 Description
The agentic Duplicate Surfacing workflow (#6399 / #6400) reasons about duplicates semantically, but its only retrieval mechanism is GitHub's full-text issue search, whose relevance ranking buries older, sparsely-worded issues. Genuine duplicates that never surface in search can never be judged by the model.
Concrete miss (found by @Trenly, #6406): on #6405 ("Support source type detection") the agent ran jargon-heavy queries (
source add --type PreIndexedPackage Rest autodetect, etc.) and no-op'd — but #1477 ("The type of a source should be automatically detected when possible") is a near-verbatim duplicate (same feature and same proposed/information-endpoint mechanism). GitHub full-text search never returned #1477, even for its own title words; onlyin:titleconcept searches surface it.This PR updates the workflow's "How to search" guidance to raise recall:
in:titleconcept searches (2–3 core nouns).The change is prompt-only — detection thresholds, safe-outputs, permissions, and action pins are unchanged.
Authored with GitHub Copilot assistance.
🔗 References
🔍 Validation
gh aw compile duplicate-surfacing→ 0 errors, 0 warnings.gh aw trialreplaying Support source type detection #6405 against the live winget-cli corpus: with the new guidance the agent surfaces The type of a source should be automatically detected when possible. #1477 with HIGH confidence (label + comment), exactly the duplicate previously missed.✅ Checklist
📋 Issue Type
Microsoft Reviewers: Open in CodeFlow