fix(copilot-session-insights): restore transcript fetch after 42-day gap

[Copilot]

Copilot session transcript logs have been empty for 42 consecutive days, causing behavioral analytics (loop detection, context confusion) to fall back to run-metadata-only inference.

Two bugs identified:

Bug 1 — branch name extraction broken since ~Jul 8 (primary)
Session numbers were extracted by stripping non-numerics from branch names. Copilot switched from copilot/issue-123 to descriptive slugs (copilot/fix-mcp-gateway-docker-daemon-access), producing empty strings that caused [ -n "$session_number" ] to skip all 50 sessions — zero files downloaded.

Bug 2 — auth failure (older, pre-Jul 8)
gh agent-task view --log doesn't have a --log flag and requires an OAuth token. Cached files from May–July contained "this command requires an OAuth token" error messages rather than transcript data.

Changes

• shared/copilot-session-data-fetch.md — replaces the broken loop with a gh api job-log approach: filters sessions-list.json to actual agent runs (conclusion != "action_required", skipping ~47 CI gate runs per day), fetches job logs via gh api repos/$REPO/actions/jobs/$job_id/logs (works with standard GITHUB_TOKEN), then greps for [cca-engine] turn= lines as the transcript

  # Before (broken): extracted session number from branch slug → always empty
  session_number=$(echo "$branch" | sed 's/copilot\///' | sed 's/[^0-9]//g')
  if [ -n "$session_number" ]; then  # always false for descriptive slugs
  
  # After: use run_id directly, skip CI gate runs, fetch via API
  jq -r '.[] | select(.conclusion != "action_required") | "\(.id) \(.head_branch)"' sessions-list.json \
    | while read -r run_id branch; do
        job_id=$(gh api "repos/$GITHUB_REPOSITORY/actions/runs/${run_id}/jobs" --jq '.jobs[0].id')
        gh api "repos/$GITHUB_REPOSITORY/actions/jobs/${job_id}/logs" \
          | grep "\[cca-engine\] turn=" > "${run_id}-conversation.txt"

• copilot-session-insights.md — adds grep * and wc * to the bash tool allow-list; updates the prompt's transcript format description to match [cca-engine] turn= line structure

• scripts/check-stale-lock-files.sh — fixes a false failure: shared components under shared/ and skills/ compile into parent workflow lock files rather than producing standalone .lock.yml files; these paths are now excluded from the stale-lock check

[github-actions]

PR Triage

Category: bug | Risk: medium | Priority: medium | Score: 48/100 (impact:25, urgency:15, quality:8) | Action: batch_review | Batch: pr-batch:draft-review

Restores transcript fetch after 42-day data gap by fixing branch-name extraction regression (descriptive slug to session-number). Medium impact/urgency. Still a draft; grouped with other draft PRs.

Generated by 🔧 PR Triage Agent · 31.2 AIC · ⌖ 8.39 AIC · ⊞ 5.4K · ◷

[github-actions]

Great detective work identifying and fixing the 42-day transcript fetch gap! 🔍 The root cause analysis covering both the branch name extraction bug and the auth failure is thorough and well-documented.

One thing that would help:

• Add tests — the new gh api job-log approach in shared/copilot-session-data-fetch.md and the stale-lock-file fix in scripts/check-stale-lock-files.sh would benefit from test coverage to prevent regressions.

If you'd like a hand, you can assign this prompt to your coding agent:

Add test coverage for the transcript fetch changes in shared/copilot-session-data-fetch.md:
1. Test that runs with conclusion != "action_required" are included
2. Test that the [cca-engine] turn= grep pattern extracts correct lines
3. Test the check-stale-lock-files.sh exclusion of shared/ and skills/ paths

Generated by ✅ Contribution Check · 141.9 AIC · ⌖ 14.9 AIC · ⊞ 6.2K · ◷