Skip to content

feat(test): add --output (text/json/yaml/github-actions) for machine-readable results#488

Open
Caesarsage wants to merge 6 commits into
microcks:masterfrom
Caesarsage:feat/test-output
Open

feat(test): add --output (text/json/yaml/github-actions) for machine-readable results#488
Caesarsage wants to merge 6 commits into
microcks:masterfrom
Caesarsage:feat/test-output

Conversation

@Caesarsage

Copy link
Copy Markdown
Contributor

Adds --output to microcks test, with four formats:

Value Output
text (default) Human-readable summary (unchanged default behaviour)
json Full TestResult as JSON
yaml Full TestResult as YAML
github-actions GitHub Actions annotations + log groups + job step summary

Works for both the normal server path and --dry-run, since it's wired into the
shared test engine (runTestAndWait).

github-actions format

  • ::group:: / ::endgroup:: log-folding per operation
  • ::error:: per failing step, with the validation message
  • ::error file=<spec>,line=<n>:: when the local spec is known (--dry-run --artifact),
    mapping the failing operation to its line so the annotation lands on the diff
  • ::notice:: for passing operations, gated by MICROCKS_ACTIONS_VERBOSE
  • A per-operation markdown table appended to $GITHUB_STEP_SUMMARY

Tests

  • Unit tests for all four formatters, the github-actions escaping, the step
    summary, and the OpenAPI line-mapping (pkg/output).

Notes

Signed-off-by: caesarsage <destinyerhabor6@gmail.com>
…ctions)

Signed-off-by: caesarsage <destinyerhabor6@gmail.com>
…paths

Signed-off-by: caesarsage <destinyerhabor6@gmail.com>
Signed-off-by: caesarsage <destinyerhabor6@gmail.com>
Signed-off-by: caesarsage <destinyerhabor6@gmail.com>
Signed-off-by: caesarsage <destinyerhabor6@gmail.com>
@Caesarsage

Copy link
Copy Markdown
Contributor Author

/hold

Working on a structure logger and helper function from last mentorship call

"encoding/json"

"github.com/microcks/microcks-cli/pkg/connectors"
"gopkg.in/yaml.v2"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In openapi_linemap file, you used yaml.v3, so why v2 here?

// within an OpenAPI spec file, or 0 if it can't be determined (non-YAML spec,
// parse error, or operation not found). yaml.v3 nodes carry line numbers, which
// yaml.v2 does not expose.
func operationLine(specPath, operationName string) int {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: the function name operationLine and the paths key lookup hard-code an assumption that this is an OpenAPI spec. But microcks test --dry-run also works with AsyncAPI, Postman Collections, and GraphQL artifacts.

For AsyncAPI, the top-level key is channels (AsyncAPI 2.x) or channels/operations (AsyncAPI 3.x), not paths. Postman Collections are JSON, not YAML at all.

if pathNode == nil {
return 0
}
line, _ := mappingEntry(pathNode, method)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The blank identifier _ here silently discards whether the method lookup found anything. If mappingEntry returns (0, nil) for a missing key, that's already handled correctly since you return 0.

But would it be worth extracting a named variable for clarity?

line, methodNode := mappingEntry(pathNode, method)
if methodNode == nil {
    return 0
}
return line

}

// splitOperation parses "GET /products" into ("get", "/products", true).
func splitOperation(operationName string) (method, path string, ok bool) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works great for the OpenAPI METHOD /path convention, but Microcks also supports AsyncAPI (e.g. SUBSCRIBE channel) and GraphQL (e.g. getPastries) where the split won't match, though the safe false fallback handles that correctly.

Worth adding a short doc comment to make this intentional scope explicit.

@Caesarsage

Copy link
Copy Markdown
Contributor Author

if #489 or #490 or #491 gets merge, I will rework this PR to use the new error format

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants