Skip to content

test: tag Raincloud corpus tests @Tag("raincloud"), excluded by default#289

Merged
dfa1 merged 1 commit into
mainfrom
test/tag-raincloud-integration-tests
Jul 19, 2026
Merged

test: tag Raincloud corpus tests @Tag("raincloud"), excluded by default#289
dfa1 merged 1 commit into
mainfrom
test/tag-raincloud-integration-tests

Conversation

@dfa1

@dfa1 dfa1 commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Summary

  • A plain ./mvnw verify silently ran the full real-world Raincloud corpus whenever a developer had hydrated it locally: the two Raincloud tests only gated real work behind assumeTrue(Files.exists(manifestPath())), a no-op skip on CI but a surprise on a dev machine. RaincloudSizeComparisonIntegrationTest in particular does a full Parquet→Vortex re-encode (FSST training, etc.) per slug, costing a release preflight 15+ minutes of unexpected work.
  • Tags both classes @Tag("raincloud"); Failsafe excludedGroups (bound to a new vortex.it.excludedGroups property, default raincloud) excludes them from test discovery entirely — stronger than assumeTrue's runtime skip.
  • Failsafe 3.5.6 applies excludedGroups even to an explicitly named -Dit.test=Raincloud... class, and a literal pom value can't be cleared from the CLI — hence the property indirection. Opting in requires -Dvortex.it.excludedGroups= in addition to -Dit.test. Documented in CLAUDE.md and the hydrate script's usage/output hints.
  • No CHANGELOG entry: test/tooling-only churn, no shipped-artifact or wire-behavior change.

Test plan

  • Rebased onto current main (post-0.12.3 release + dependabot bumps + ParquetImporter fix) and re-verified after rebase
  • ./mvnw verify -pl integration -am (corpus hydrated locally): neither Raincloud class discovered/run; 294 tests, 0 failures, ~1:25 total (previously 20+ min)
  • Opt-in (-Dvortex.it.excludedGroups= -Dit.test=RaincloudConformanceIntegrationTest): full conformance run passes end-to-end against the real corpus
  • ./mvnw test -pl integration -am: no regressions

🤖 Generated with Claude Code

A plain `./mvnw verify` silently ran the full real-world Raincloud corpus
whenever a developer had hydrated it locally (via hydrate-raincloud-corpus.sh):
the two Raincloud tests only gated real work behind
`assumeTrue(Files.exists(manifestPath()))`, which is a no-op skip on CI (corpus
never hydrated) but a surprise on a dev machine. In particular
RaincloudSizeComparisonIntegrationTest — documented as purely informational —
does a full Parquet->Vortex re-encode (FSST training, etc.) per slug, costing a
release preflight 15+ minutes of unexpected CPU-heavy work.

Make the exclusion explicit and controllable with JUnit 5 `@Tag`:
- Tag both classes `@Tag("raincloud")`.
- Failsafe `excludedGroups` (integration/pom.xml), bound to a new
  `vortex.it.excludedGroups` property defaulting to `raincloud`, so a routine
  build excludes them from test discovery entirely (stronger than assumeTrue's
  skip) yet the CLI can opt back in.

Verified against a locally hydrated 10-slug corpus:
- `./mvnw verify -pl integration -am` no longer discovers/runs either class.
- Failsafe 3.5.6 applies `excludedGroups` even to an explicitly named
  `-Dit.test=Raincloud...` class, and a literal pom value cannot be cleared
  from the CLI — hence the property indirection. Opting in requires
  `-Dvortex.it.excludedGroups=` in addition to `-Dit.test`. The full
  conformance run (11 tests) passes end-to-end with that flag.

Updated the documented invocations to include the flag: CLAUDE.md Commands
section and the hydrate script's usage/output hints. No CHANGELOG entry:
test/tooling-only churn, no change to shipped artifacts.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@dfa1
dfa1 force-pushed the test/tag-raincloud-integration-tests branch from 5262dc3 to 6fbdb1e Compare July 19, 2026 16:44
@dfa1

dfa1 commented Jul 19, 2026

Copy link
Copy Markdown
Owner Author

Ran the `review` skill against this PR before merging. It found the pom.xml/CLAUDE.md/hydrate-script core mechanism correct, but caught a real bug: two call sites documenting the old (now-broken) invocation weren't updated, and both test classes' javadoc had a wrong property name plus a claim that directly contradicts the pom.xml comment added in the same diff.

Fixed (amended into the PR commit):

  1. .github/workflows/raincloud-conformance.yml — the weekly scheduled job ran -Dit.test=RaincloudConformanceIntegrationTest without -Dvortex.it.excludedGroups=; after merge it would have silently run 0 tests and stayed green, disabling the entire conformance signal.
  2. docs/compatibility.md — same stale command.
    3/4. Both test classes' javadoc used the wrong property name (-DexcludedGroups=, a no-op) and RaincloudConformanceIntegrationTest's javadoc falsely claimed explicit -Dit.test selection overrides the tag exclusion.

Re-verified after fixing: plain `verify` still excludes both classes (294 tests, 0 failures, ~1min), and the corrected opt-in command (`-Dvortex.it.excludedGroups= -Dit.test=RaincloudConformanceIntegrationTest`) now actually runs the test (confirmed via the failsafe report: `Tests run: 1`).

@dfa1
dfa1 merged commit 4dee0b7 into main Jul 19, 2026
6 checks passed
@dfa1
dfa1 deleted the test/tag-raincloud-integration-tests branch July 20, 2026 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant