Skip to content

feat: consolidate pipeline validation#30

Merged
Volv-G merged 1 commit into
masterfrom
piforge/pplx-pr-review/consolidate-pipeline-validation-0f07e91
Jul 15, 2026
Merged

feat: consolidate pipeline validation#30
Volv-G merged 1 commit into
masterfrom
piforge/pplx-pr-review/consolidate-pipeline-validation-0f07e91

Conversation

@Volv-G

@Volv-G Volv-G commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

Consolidates Tangle pipeline validation into OSS tangle-cli so local authoring and submit paths share the same checks:

  • tangle sdk pipelines validate
  • tangle sdk pipeline-runs submit / --dry-run
  • PipelineRunner.run_pipeline

The shared validator now covers graph shape, the packaged Tangle pipeline JSON schema, and component-input wiring when component specs are available. Submit validation remains post-hydration by default; --no-hydrate validates the as-is spec.

Motivation

tangle-deploy had duplicated generic pipeline validators that belong in the OSS CLI. Moving the generic validation surface here lets tangle-cli reject malformed pipelines consistently before API submission while avoiding runtime schema fetching or executing upstream source code.

What changed

  • Added canonical validation module tangle_cli.pipeline_validation for:
    • graph/root/task shape checks
    • vendored JSON schema validation
    • component-input wiring validation
  • Added shared spec traversal helper module tangle_cli.pipeline_spec_utils.
  • Kept backward-compatible re-exports from tangle_cli.pipelines so existing imports continue to work.
  • Vendored tangle_cli/schemas/pipeline_schema.json and added scripts/refresh_pipeline_schema.py for manual regeneration.
    • No runtime network fetch.
    • No runtime exec() of upstream schema code.
  • Added jsonschema as a core tangle-cli dependency.
  • Unified submit validation at the base PipelineRunManager choke point.
  • Deduplicated PipelineRunner validation so high-level runner usage validates once, while direct/base submit paths still validate once.
  • Added docstrings for the validation module and helpers.
  • Bumped tangle-cli from 0.1.3 to 0.1.4.

Test plan

  • uv run --frozen pytest -q705 passed, 4 warnings
  • uv run --frozen ruff check scripts/refresh_pipeline_schema.py packages/tangle-cli/src/tangle_cli/__init__.py packages/tangle-cli/src/tangle_cli/pipeline_run_manager.py packages/tangle-cli/src/tangle_cli/pipeline_runner.py packages/tangle-cli/src/tangle_cli/pipelines.py packages/tangle-cli/src/tangle_cli/pipeline_spec_utils.py packages/tangle-cli/src/tangle_cli/pipeline_validation.py tests/test_packaging.py tests/test_pipeline_runs_cli.py tests/test_pipelines_cli.py → passed
  • git diff --check → passed
  • Blind review by a separate reviewer found 1 blocking issue and 2 should-fix gaps; all were fixed and re-verified:
    • unknown producer outputs no longer false-reject unhydrated/--no-hydrate graphs
    • supplied optional/defaulted inputs now get reference validation
    • default: null now satisfies default semantics consistently with submit conversion

Notes / follow-ups

  • A follow-up Discovery PR will bump the submodule to this release and remove tangle-deploy's now-redundant generic validators.
  • Container-image existence validation is intentionally deferred to a later opt-in PR.

Move graph-shape, vendored JSON schema, and component-input validation into tangle-cli so pipeline validation, pipeline-run submit/dry-run, and PipelineRunner share one validation surface. Deduplicate validation so each path validates once, add the vendored schema/refresh helper and jsonschema dependency, preserve tangle_cli.pipelines re-exports for back-compat, and bump tangle-cli to 0.1.4.

Assisted-By: devx/5490f7cb-c7e4-4cf5-b525-a8c4e2af55b1
@Volv-G
Volv-G requested a review from Ark-kun as a code owner July 15, 2026 19:24
@Volv-G
Volv-G merged commit 57b8cfa into master Jul 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant