Skip to content

fix(release): v2.3.9 Engine artifacts embed version 2.3.7 #5545

Description

@oatiz

Summary

The official v2.3.9 Engine artifacts embed Cargo version 2.3.7.

This is not only stale source metadata: the npm package published as @rivetkit/engine-cli-darwin-arm64@2.3.9 runs as Rivet 2.3.7, and a self-hosted rivetdev/engine:full-2.3.9 deployment reports 2.3.7 from /health.

Reproduction

tmp_dir=$(mktemp -d)
cd "$tmp_dir"
tarball=$(npm pack --silent @rivetkit/engine-cli-darwin-arm64@2.3.9)
tar -xzf "$tarball"
./package/rivet-engine --version

Actual:

Rivet 2.3.7

The tagged source has the same mismatch:

git show v2.3.9:Cargo.toml   | python3 -c 'import sys, tomllib; print(tomllib.load(sys.stdin.buffer)["workspace"]["package"]["version"])'
# 2.3.7

engine/packages/engine/Cargo.toml uses version.workspace = true. Both the health response and the startup rollback check read CARGO_PKG_VERSION, so this value reaches runtime behavior, not only package metadata:

Release workflow ordering

The successful 2.3.9 release run built the Engine binaries and Docker images from commit c009b34 before changing the Cargo version:

  • Release run: https://github.com/rivet-dev/rivet/actions/runs/30023397613
  • Engine/Docker builds completed between 16:06 and 16:19 UTC.
  • The Publish job changed Cargo.toml to 2.3.9 at 16:21 UTC, after those artifacts already existed.
  • The Publish job then packaged the prebuilt binaries and retagged the Docker manifests as 2.3.9.

The workflow source shows the same ordering: Docker images are built before the Publish job's bump-versions step.

Impact

  • --version, /health, metadata, metrics, and user agents identify a 2.3.9 build as 2.3.7.
  • verify_engine_version persists and compares the wrong version for rollback protection.
  • Self-hosters cannot reliably verify the required Engine/RivetKit version lockstep.
  • This surfaced while diagnosing an agentOS Apps connection failure: the app/RivetKit packages were 2.3.9 while the Engine image tagged 2.3.9 reported 2.3.7. The incorrect identity obscures compatibility diagnosis; I am not claiming the version string alone causes the Envoy rejection.

This is distinct from #5484, which covered Envoy protocol negotiation and Rivet Cloud rollout compatibility.

Expected behavior

  • Every Rust/Engine artifact should be compiled after the workspace version is set to the requested release version.
  • The release workflow should fail before publishing or retagging if rivet-engine --version does not equal the release input/tag.
  • Since released artifacts should not be replaced in place, please document or otherwise handle the affected 2.3.9 artifacts and prevent recurrence in a subsequent release.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions