Skip to content

test: close test framework parity gaps - #47

Merged
Pluze merged 34 commits into
mainfrom
codex/test-framework-parity-closure
Jul 24, 2026
Merged

test: close test framework parity gaps#47
Pluze merged 34 commits into
mainfrom
codex/test-framework-parity-closure

Conversation

@Pluze

@Pluze Pluze commented Jul 24, 2026

Copy link
Copy Markdown
Owner

Goal and scope

  • Goal: close the behavioral parity debt from the catalog cutover and remove migration-era bypasses without restoring legacy runners, wrappers, selectors, or test-owned product APIs.
  • In scope: owner-first App and framework evidence, path-isolated App conformance, complete changed-file fallback, aggregated failure reporting, production-source routing guardrails, bounded ManualChecks, structural GUI conformance, representative App workflows, repository hygiene, and contract-facade compatibility behavior.
  • Intentionally out of scope: native dialog feel, pointer feel, visual design review, real lab data, and scientific interpretation; these remain explicit non-passing manual checks.

User-visible behavior

  • Before: a green catalog did not prove the high-value legacy behavior set; GUI evidence was largely startup smoke; unknown changes omitted GUI/path-isolation evidence; an isolation helper started a second MATLAB; and an agent skill still exposed the deleted runner model.
  • After: catalog, plan, and run remain the only execution path; all audited App rows have current direct or bounded workflow evidence; every production MATLAB source has an explicit evidence location; path isolation runs inside the unified executor; and active guidance/automation contains no legacy runner wrapper.
  • Compatibility / upgrade note: test and maintainer-tooling migration only. App algorithms, saved project formats, exports, and public LabKit APIs are unchanged.

Validation

  • I ran the smallest relevant local validation from docs/development/maintain-and-release/testing.md.
  • I ran the required final gate for this handoff, or this PR is not ready for final review.
  • Automated validation was not run; reason and risk are recorded below.

Commands and results:

framework/contract source: 4 passed
system/build system: 30 passed
system/repository system: 3 passed
apps/conformance hidden-gui: 21 passed
isolated profile: 1 aggregate identity passed for all 21 public Apps
docs + docsCheck: 218 narrative pages, 149 API pages, 372 generated files current
changedFast full branch gate: 298 passed (258 headless, 39 hidden-GUI, 1 path-isolated)
FocusStack fixture changedFast: 2 passed (presentation + hidden-GUI workflow)
PR CI: all 9 Linux, macOS, and Windows jobs passed

Manual / native behavior not verified:

  • Native dialogs, pointer feel, visual design quality, real-data suitability, and scientific interpretation. Mapped layouts emit concrete non-passing ManualChecks for these boundaries.

Documentation and boundaries

  • Human docs were updated where user-facing behavior or public contracts changed.
  • Agent guidance was updated where execution, ownership, or governance rules changed.
  • Docs and agent guidance were left unchanged because their contracts were preserved.
  • Versioned component docs and a related docs/history/records/**/*.md record were updated where required.
  • App-specific workflow logic stayed in the owning app.
  • Reusable +labkit changes, if any, are domain-neutral and app-facing.

Boundary decision:

  • No product-facing API was added. The temporary test-only DefinitionInspector method was removed; the conformance spec uses the existing private inspection boundary.

Delivery state

  • The branch is committed and pushed.
  • Required GitHub checks are linked below, or are pending:

CI / review links:

Data hygiene

  • This PR does not add raw lab files, identifying file names, subject names, device serials, local absolute paths, timestamps, or private experiment labels.

Unverified behavior

  • Automated hidden-GUI checks do not prove native dialog behavior, pointer feel, visual quality, real-data suitability, or scientific interpretation.

@Pluze
Pluze marked this pull request as ready for review July 24, 2026 06:39
@Pluze
Pluze merged commit c4e4462 into main Jul 24, 2026
9 checks passed
@Pluze
Pluze deleted the codex/test-framework-parity-closure branch July 24, 2026 06:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant