Skip to content

Malformed executionOrder is silently ignored — sequential flows run in parallel with no warning #110

Description

@finalerock44

Problem

A config.yaml whose executionOrder is the wrong shape is silently ignored, and every flow runs in parallel instead of in sequence. There is no warning, no error, and the run goes green — so the only way to notice is to inspect depends_on on the result rows.

The intuitive-but-wrong form:

# silently runs a, b, c in PARALLEL
executionOrder:
  - a.yaml
  - b.yaml
  - c.yaml

The correct form (documented here):

executionOrder:
  continueOnFailure: true
  flowsOrder:
    - a.yaml
    - b.yaml
    - c.yaml

To be clear: the bare-list form is not valid Maestro either, so this isn't a compatibility divergence. The bug is that an invalid config fails silently rather than being reported.

Why it matters

  • Same cost, wrong semantics. The user is billed identically and gets a green run they believe was ordered.
  • For a suite where later flows depend on earlier state (login → browse → checkout), parallel execution surfaces as confusing, flaky-looking assertion failures rather than as "your config is wrong".
  • It is silent in the one direction that matters. Sequencing is usually adopted because ordering is required.

Root cause

resolveSequentialFlows bails when flowsOrder is missing, without distinguishing "no executionOrder at all" (fine) from "executionOrder present but malformed" (a mistake worth reporting):

https://github.com/devicecloud-dev/dcd-cli/blob/main/src/services/execution-plan.service.ts#L315

if (!workspaceConfig.executionOrder?.flowsOrder) {
  return [];
}

With a bare list, executionOrder is an Array, so .flowsOrder is undefined and this returns [].

Nothing upstream catches it either — the config is yaml.loaded and straight-cast, with no runtime validation (execution-plan.service.ts:233):

workspaceConfig = readYamlFileAsJson(configFilePath) as IWorkspaceConfig;

IExecutionOrder is a compile-time interface only, so the cast asserts a shape that was never checked.

The telling asymmetry

There is already a warning for the adjacent mistake — a flow name in flowsOrder that matches nothing (execution-plan.service.ts:342-353), and the docs advertise it:

If a flow name in flowsOrder doesn't match any discovered flow (e.g. after tag filtering), the CLI will emit a warning and lists the available names to help diagnose the mismatch.

So we warn when the contents are wrong but stay silent when the shape is wrong — and the shape mistake is the easier one to make.

Worth noting readYamlFileAsJson already normalizes other fields defensively, coercing a scalar includeTags/excludeTags into an array (execution-plan.utils.ts:70-78). executionOrder getting no such treatment is an inconsistency in the same loader, and suggests a natural home for the fix.

Suggested fix

Either is fine; they compose:

  1. Warn (or error) on a malformed executionOrder. Detect executionOrder present but not an object with an array flowsOrder, and say so, naming the expected shape. Cheapest, and matches the existing warning's tone.
  2. Normalize the bare list, the way includeTags is normalized — treat executionOrder: [a, b] as { flowsOrder: [a, b] }. More forgiving, but it accepts a form Maestro rejects, so 1 may be preferable.
  3. Validate the workspace config with zod, which is already a dependency. Catches this and the whole class of config typos (flowOrder, executionorder, …) in one place.

Reproduction

Found while doing dev verification for dcd#920. Two submissions, identical except for the executionOrder shape:

config shape upload result rows depends_on
bare list 75c4f464-4f71-4ca7-85f2-1f408e5dad4e 36837, 36838, 36839 null, null, null — ran in parallel
flowsOrder f0377349-3ba4-4bf3-9d6b-76a2d1ac67ac 36842, 36843, 36844 null, 36842, 36843 — correctly chained

Both were accepted, billed the same ($0.24), and reported success. The API behaved correctly in both cases — it faithfully ran what the CLI sent it. The flows were simply never submitted as sequential.

Steps:

  1. Create a workspace with a.yaml, b.yaml, c.yaml and a config.yaml using the bare-list executionOrder.
  2. dcd cloud --app <apk> --async <workspace>
  3. Observe "Created 3 tests" with no warning, and depends_on IS NULL on all three rows.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions