Skip to content

ci(#553): split the CI quality gates into parallel reusable workflows - #556

Merged
drmoisan merged 13 commits into
mainfrom
feature/ci-parallel-job-split-553
Aug 15, 2026
Merged

ci(#553): split the CI quality gates into parallel reusable workflows#556
drmoisan merged 13 commits into
mainfrom
feature/ci-parallel-job-split-553

Conversation

@drmoisan

Copy link
Copy Markdown
Owner

Split the CI quality gates into parallel reusable workflows

Summary

  • Decomposes .github/workflows/ci.yml from a 160-line file with one monolithic windows-latest job into a 32-line orchestrator that calls five reusable workflows with no inline steps: and no needs: edges, so the four quality gates run concurrently instead of serially.
  • Measured wall clock drops from 444s to a median 259s (a 41.7% reduction) across four post-split samples (259s, 433s, 245s, 296s). The 444s baseline is measured, not estimated.
  • Every gate run: block was transplanted byte-identically, verified by SHA-256 comparison of 14 extracted step blocks and independently re-verified during review.
  • Failure isolation was proven empirically by three pushed-and-reverted probes, not asserted: each violation turned exactly one gate red while the other four reported independently.
  • The main branch ruleset has already been migrated (see Backward Compatibility). Both previously required contexts were retired by this change and have been replaced with five per-gate contexts.
  • Estimated billed windows-latest seconds rise from ~444 to ~764, roughly 1.7x, before GitHub's Windows 2x multiplier. This is the deliberate cost of the latency reduction.

Why

ci.yml ran the entire C# quality toolchain inside a single quality-gates job. Wall-clock latency was the sum of all stages, and the measured baseline (green run 31749877507) breaks down as:

Phase Duration
Fixed setup (checkout, SDK, MSBuild, NuGet, caches, restore) 130s
Verify formatting 15s
Build with analyzers 101s
Build with nullable warnings as errors (/t:Rebuild) 98s
MSTest with coverage 88s
Total 444s

Two consequences beyond latency:

  • No independent failure signal. All gates reported as one required check named Format, build, analyze, and test, so a red check did not identify which gate failed without opening the log.
  • No independent re-dispatch. A transient failure in one stage required re-running the entire job.

What Changed

CI pipeline (7 files, +487 / -141)

File Change
ci.yml Rewritten as a 32-line orchestrator: on, permissions, concurrency, and five uses: jobs. No inline steps:, no needs:.
_actionlint.yml New. ubuntu-latest, 10-minute timeout.
_format-check.yml New. CSharpier gate. Drops setup-msbuild, setup-nuget, the packages cache, and nuget restore — CSharpier reads source text only.
_build-analyzers.yml New. /t:Build with EnableNETAnalyzers and EnforceCodeStyleInBuild.
_build-nullable.yml New. /t:Rebuild with TreatWarningsAsErrors, including its 7-line in-file rationale comment.
_mstest-coverage.yml New. Plain build, then the unchanged discovery filter and vstest invocation, then the test-results upload.
README.md New. Documents per-stage workflow_dispatch and the branch-protection rename procedure.

Each callee declares both on: workflow_call: and on: workflow_dispatch:, its own permissions: contents: read, a right-sized timeout-minutes, and no concurrency block — the caller owns the concurrency group.

Documentation and evidence

The remaining 60 files are the feature folder for issue #553: research, spec, user story, the atomic plan, review artifacts, and the evidence tree. Two files under docs/features/potential/promoted/ are unrelated archival records for issues #554 and #555 (latent defects found while doing this work) and were committed separately.

Architecture / How It Fits Together

ci.yml retains the triggers, permissions, and the workflow-level concurrency group, then fans out to five independent jobs. Because a called workflow's jobs run as part of the caller's run, the existing group continues to cover all five, and cancel-in-progress: true cancels them together when a newer run supersedes.

There are no needs: edges. Build-output sharing between jobs was evaluated and rejected: a needs: edge serializes the build job's setup and compile ahead of the test job, so it loses to a per-job rebuild on critical-path arithmetic even at zero transfer cost, and it adds .pdb/dependency-closure fragility.

The analyzer and nullable compiles were deliberately not merged. A single invocation carrying both property sets would either promote analyzer warnings to errors (an unratified strengthening) or require WarningsNotAsErrors carve-outs that risk exempting nullable diagnostics (a weakening). Either direction alters a gate, so both compiles run unchanged in separate jobs.

Verification

Completed

  • Byte-identity: 14 step blocks SHA-256-compared between the pre-split ci.yml and the callees; 14/14 match, including both if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE } guards, the vstest discovery filter, the zero-assembly throw, the /t:Rebuild rationale comment, and the test-results upload with if: always().

  • actionlint: exit 0 over all seven workflow files, confirmed by -verbose naming each file individually.

  • Green runs: multiple, including 31840944277 at the current head with all five jobs success.

  • Fault isolation, empirically proven:

    Probe Violation Red gate Other gates
    Format Indentation changed in a .cs file format-check only all success
    Nullable => null; in a #nullable enable file (CS8603) build-nullable only all success
    MSTest Inverted a FluentAssertions assertion mstest-coverage only (1 of 6435 tests) all success

    The nullable probe is the sharpest evidence of preserved gate semantics: identical source produced only a warning in the analyzer job and in the MSTest job's plain build, and an error only under the nullable gate's TreatWarningsAsErrors.

  • Probe cleanliness: all three probe/revert pairs are byte-exact; the net branch diff contains zero C# or project files.

  • $LASTEXITCODE hygiene: every pwsh step reviewed against .claude/rules/ci-workflows.md. None uses the deliberately-failing nested-command pattern, so no explicit reset is required.

  • Review: cycle-2 feature review reports zero blocking findings and 18 of 18 acceptance criteria passing.

  • No C# toolchain run. No C# or project file changed on this branch, so csharpier/msbuild/vstest were not run locally; the gates themselves exercise them in CI.

Recommended

gh workflow run _format-check.yml     --ref main
gh workflow run _build-analyzers.yml  --ref main
gh workflow run _build-nullable.yml   --ref main
gh workflow run _mstest-coverage.yml  --ref main
gh workflow run _actionlint.yml       --ref main
gh api repos/drmoisan/TaskMaster/rulesets/18572843 --jq '.rules[] | select(.type=="required_status_checks")'

Backward Compatibility / Migration Notes

The main ruleset (id 18572843) was migrated before this PR was opened, with explicit owner authorization. A reviewer should not be surprised by it.

Before After
Required contexts actionlint, Format, build, analyze, and test actionlint / actionlint, format-check / Verify formatting, build-analyzers / Build with analyzers and code style enforcement, build-nullable / Build with nullable warnings treated as errors, mstest-coverage / Run MSTest suite with coverage
strict_required_status_checks_policy true true (unchanged)
Rule types 4 4 (unchanged)

Both old contexts were retired by this change: the monolithic job was split, and actionlint became actionlint / actionlint when it moved into a callee, because a called workflow reports as <caller job id> / <callee job name>.

The migration was a single atomic PUT of the full writable object. Five pre-PUT checks passed, including a diff-confinement check proving the payload altered only the contexts array and could not silently drop the deletion, non_fast_forward, or pull_request rules. Post-PUT set equality was verified by an independent GET, not the PUT response body. Evidence, including the pre-PUT object, the exact payload, and the post-PUT state, is under evidence/other/ruleset-migration/.

Consequence: every other open PR against main whose head predates this change reports the old contexts and cannot report the new ones, so it is blocked until it updates its branch past this merge. This is over-blocking, never under-gating, but it means this PR should merge promptly.

Rollback: a single PUT restoring the previous contexts set reverts the merge policy; the workflow change itself reverts as an ordinary revert PR.

Risks and Mitigations

Risk Mitigation
Billed Windows minutes rise ~1.7x because the 130s setup is paid per job Setup was tailored per job (format drops NuGet restore; msbuild jobs drop setup-dotnet), trimming ~57s per job. Verified green on first run.
Hosted-runner variance can erode the measured gain Four samples reported, including the unfavourable 433s outlier. Median 259s. The outlier is documented: every compute-bound step scaled ~1.6x while fixed costs stayed flat, with queueing excluded.
A context-name mismatch would leave a permanently unreportable required check Names were captured from live runs, never assumed, and verified set-equal against the live ruleset after the PUT.
Cross-run runner contention could queue jobs The 5-job per-run demand is below every plan's concurrent-job ceiling. Cross-run contention is not determinable from repository data.

Review Guide

Suggested order:

  1. .github/workflows/ci.yml — 32 lines, the whole orchestration.
  2. The five _*.yml callees — the substance. Compare against evidence/other/pre-split/ci.yml.pre-split.txt.
  3. evidence/qa-gates/byte-identity.2026-08-14T09-54.md — the transplant proof.
  4. evidence/regression-testing/probe-*.md — the fault-isolation proof.
  5. evidence/other/ruleset-migration/ — the branch-protection change.
  6. .github/workflows/README.md — operational procedures.

The remaining feature-folder documents are supporting material and can be skimmed. The 13 commits include three probe/revert pairs that cancel out; reviewing the net diff is more useful than walking commit by commit.

Follow-ups

  • Plan Phase 6 checkboxes lag the executed migration; reconcile in the post-merge phase (non-blocking).
  • The actionlint tarball in the _actionlint.yml step is not checksum-pinned. Pre-existing, transplanted byte-identically.
  • GitHub emits a Node.js 20 deprecation warning for actions/cache@v4, actions/checkout@v4, microsoft/setup-msbuild@v2, and nuget/setup-nuget@v2. Pre-existing, not a regression from this change.
  • Moving the format gate to ubuntu-latest (CSharpier is cross-platform; Linux bills at 1x) was deliberately deferred as a separate platform-parity question.
  • Issues Bug: potential-to-issue-promoted-copy-not-written #554 and Bug: orchestrator-hooks-reference-absent-python-validators #555 were filed for latent defects found during this work.

GitHub Auto-close

drmoisan and others added 13 commits August 14, 2026 16:59
… spec, and user story

Promotes the ci-parallel-job-split potential entry to issue #553 and creates the
active feature folder. Captures the measured sequential CI baseline from green run
31749877507 (444s wall clock; 130s fixed setup; 15s/101s/98s/88s gate durations),
records the research artifact resolving ten design questions, and authors spec.md
and user-story.md against the adopted topology.

No workflow files are modified by this commit.

Refs #553

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LZFrbwyXBnwJ44RRJV1N4E
Two latent defects were found while orchestrating issue #553 and promoted to their
own issues. These are the archival copies the promotion lifecycle writes to
docs/features/potential/promoted/.

- #554 potential_to_issue returned a success receipt naming a promoted/ destination
  path it never wrote, and removed the source entry. Observed on the feature path;
  the bug path was subsequently verified working, so the issue carries a comment
  narrowing the reproduction.
- #555 validate-orchestrator-output.ps1 invokes a Python validator module that does
  not exist in this repository, so an Agent(orchestrator) delegation would be blocked
  at SubagentStop with a misleading MODEL_ROUTING_BLOCKED reason.

Neither defect is part of the CI parallel job split. They are committed separately so
the feature commit stays confined to the pipeline change.

Refs #554, #555

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LZFrbwyXBnwJ44RRJV1N4E
…orkflows

- Rewrite .github/workflows/ci.yml as a 32-line orchestrator with five workflow_call references and no inline steps or needs edges
- Add five callee workflows (_format-check, _build-analyzers, _build-nullable, _actionlint, _mstest-coverage), each supporting workflow_call and workflow_dispatch, with run blocks transplanted byte-identically from the prior job
- Add .github/workflows/README.md documenting the split workflows
- Record baseline, byte-identity, and actionlint evidence under docs/features/active/2026-08-14-ci-parallel-job-split-553/evidence/qa-gates/ and evidence/baseline/
- Update atomic-executor and atomic-planner agent memory with pwsh/git/gh CLI gotchas and plan seams for issue #553

Refs: #553

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LZFrbwyXBnwJ44RRJV1N4E
…offs

Captures the evidence produced while validating the parallel CI split on live
runs, plus the feature-review artifacts and the two minor review-finding fixes.

Evidence added:
- qa-gates: first-run, tailored-setup-fallback (NOT REQUIRED - assumption held),
  post-probe-green-run, test-results-artifact, ci-split-timing-comparison,
  actionlint-final, lastexitcode-review, no-csharp-diff, file-size-audit
- regression-testing: the three seeded fault-isolation probes, each showing
  exactly one red gate and each fully reverted
- other: ac-checkoff-register recording the evidence pointer for every
  acceptance criterion checked off in Phase 5

Review-finding fixes:
- F2: .github/workflows/README.md used the context form "CI / <gate>". Corrected
  to the verified "<caller job> / <callee job>" form and added the five observed
  context strings verbatim, with a caution to capture rather than hand-write them.
- F3: added baseline.provenance.json beside the 444s sequential baseline and
  post-split-timing.provenance.json beside the post-split measurement, each
  recording runner_class, host_signature, and workflow_run_url per
  .claude/rules/benchmark-baselines.md.

Acceptance criteria: spec 1-5, 7, 8, 10 checked; user-story and issue mirrors
updated. Spec AC 6 and 9 (and their mirrors) remain open pending the ruleset PUT,
which is orchestrator-gated. Seeded conditions 1 and 3-8 checked; condition 2
awaits the post-merge standalone dispatch smoke.

No C# source, project, or build file is changed by this branch; verified by
no-csharp-diff against the merge base.

Refs #553
…g addendum

Captures the final pre-migration evidence for the CI parallel job split:

- check-run-names: the five context strings observed on the branch head, confirming
  the '<caller job id> / <callee job name>' form. Both previously required contexts
  are obsolete: 'Format, build, analyze, and test' and the bare 'actionlint', which
  became 'actionlint / actionlint' when it moved into a callee.
- pre-migration-green: the green run recorded immediately before the ruleset
  migration.
- timing addendum: adds a third sample and reclassifies run 31812508684 as a
  hosted-runner outlier. Three samples measured 259s, 433s, and 245s against the
  444s sequential baseline; median 259s, a 41.7% reduction.

Workflow files are unchanged by this commit, so the five check-run contexts are
unaffected by the head advancing.

Refs #553

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LZFrbwyXBnwJ44RRJV1N4E
Documents the single atomic PUT that replaced the main ruleset's required status
check contexts (id 18572843), performed with explicit owner authorization.

Before (2 contexts): actionlint; Format, build, analyze, and test
After (5 contexts):  actionlint / actionlint; format-check / Verify formatting;
                     build-analyzers / ...; build-nullable / ...; mstest-coverage / ...

Both prior contexts were obsolete: the monolithic job was split, and actionlint was
renamed to 'actionlint / actionlint' by moving into a callee workflow.

Five pre-PUT checks passed, including a diff-confinement check proving the payload
altered only the contexts array and could not silently drop the deletion,
non_fast_forward, or pull_request rules. Post-PUT set equality verified by an
independent GET rather than the PUT response body. strict_required_status_checks_policy
remains true throughout, so no window existed in which main could be merged unguarded.

Captures ruleset-pre.json, the exact ruleset-new.json payload, and ruleset-post.json.

Refs #553

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LZFrbwyXBnwJ44RRJV1N4E
… check-off

Cycle-2 feature review against the true merge base 35e0289 reports zero blocking
findings. All 18 acceptance criteria in spec.md and user-story.md are now checked off
with verified evidence.

The re-audit corrected two coordinator errors: the supplied merge base was stale
(the branch was rebased onto main after PR #552 merged), and the cited green run at
d83bf37 did not cover the current head, since that SHA is not an ancestor of it. The
reviewer verified the workflow bytes were byte-identical across the rebase and then
dispatched run 31840944277 at the true head, which passed all five jobs.

Independently verified during the re-audit: all three probe pairs revert byte-exactly
with zero C# or project files in the net branch diff; each probe turned exactly one
gate red, empirically proving per-gate failure isolation; the ruleset payload was a
writable-fields-only projection with a contexts-array-only delta; and a live GET of
ruleset 18572843 matches the committed post-PUT evidence on every material field.

Refs #553

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LZFrbwyXBnwJ44RRJV1N4E
@drmoisan
drmoisan merged commit 0569ac0 into main Aug 15, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: ci-parallel-job-split

1 participant