Add continue_prev invariant test against latest released CLI#5859
Open
denik wants to merge 4 commits into
Open
Add continue_prev invariant test against latest released CLI#5859denik wants to merge 4 commits into
denik wants to merge 4 commits into
Conversation
Mirrors continue_293 but seeds the deploy with the latest released CLI instead of v0.293.0. Since the previous release supports nearly every resource feature, only two configs are excluded (volume_path_job_ref for #5550 and postgres_project for #5782), plus the 1000-task scale case already covered by no_drift. Co-authored-by: Isaac
Contributor
Approval status: pending
|
…se-scoped Co-authored-by: Isaac
Collaborator
Integration test reportCommit: 674cad7
979 interesting tests: 975 MISS, 3 RECOVERED, 1 KNOWN
Top 8 slowest tests (at least 2 minutes):
|
Refactor DownloadCLI, resolveLatestVersion, downloadToFile, and extractFileFromZip to return errors instead of require.NoError, so they can be called from goroutines without triggering testifylint's go-require rule. On cold CI cache both binary downloads now run concurrently under an errgroup; the file-based caches keep warm runs a no-op. Also fix a pre-existing cache-poisoning bug in extractFileFromZip: write to a temp file and os.Rename into place, so a mid-copy failure can't leave a truncated binary that a later DownloadCLI would accept via os.Stat. Co-authored-by: Isaac
resolveLatestVersion used to hit api.github.com to look up the most recent released CLI tag. Replace it with a CHANGELOG.md parse that returns the second-most-recent release, and drop the 1h cache along with the network dependency. - Local: no network call, no GitHub API rate-limit exposure, no need to reach out from the CI runner. - Reproducible: the CLI_PREV version is now a property of the tree at HEAD, so every runner sees the same value. - N-2 avoids the release-workflow race window: the release commit bumps CHANGELOG.md on main before the release tag is pushed, so between those two events the top entry names a version whose archive does not exist yet. Skipping one entry back is always safe because that release is already tagged and published. Testing against v1.5.0 rather than v1.6.0 still exercises the two exclusions in continue_prev/test.toml (postgres_project #5782 and volume_path #5550 both landed after v1.5.0). Comments updated to name the N-2 constraint. Co-authored-by: Isaac
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
acceptance/bundle/invariant/continue_prev/— same shape ascontinue_293, but seeds the deploy with the latest released CLI (downloaded viaresolveLatestVersion) instead of v0.293.0.volume_path_job_ref(feature from Expose computed volume_path during initialize #5550 not yet released) andpostgres_project(bug fixed by direct: Fix state serialization for zero-value fields on deeply-embedded SDK specs #5782 not yet released). Both exclusions will drop out with the next release.CLI_PREV/[CLI_PREV]/[CLI_PREV_VERSION]wiring inacceptance_test.goso the test output is stable across releases.Test plan
go test ./acceptance -run 'TestAccept/bundle/invariant/continue_prev'— all 45 variants pass locallygo test ./acceptance -run 'TestAccept/bundle/invariant/continue_293'— still passes./task fmt-q/./task lint-qcleanThis pull request and its description were written by Isaac.