Skip to content

fix(bucket): retry the release until COSI provisions the BucketInfo Secret#3320

Draft
Andrey Kolkov (androndo) wants to merge 1 commit into
mainfrom
fix/3247-bucket-credentials-lookup
Draft

fix(bucket): retry the release until COSI provisions the BucketInfo Secret#3320
Andrey Kolkov (androndo) wants to merge 1 commit into
mainfrom
fix/3247-bucket-credentials-lookup

Conversation

@androndo

Copy link
Copy Markdown
Contributor

What this PR does

Fixes #3247. The -credentials Secret rendered by system/bucket (packages/system/bucket/templates/user-credentials.yaml) was gated on a Helm lookup of the COSI-provisioned BucketInfo Secret. COSI provisions that Secret asynchronously, after the BucketAccess is granted, and Helm re-evaluates lookup only during a real install/upgrade attempt — so when the COSI Secret appeared after the release's last render, the -credentials Secret was silently skipped and never created (nothing triggers another render; the release is otherwise unchanged). Consumers such as the platform backup flow (backupstrategy-controller projecting bucket-cozy-backups-system-credentials) then hang forever in CredentialsProjectionPending.

The fix makes the render fail instead of silently skipping while the COSI Secret is missing. The parent HelmRelease (rendered by apps/bucket) already sets install/upgrade remediation retries: -1, so helm-controller keeps retrying — and each retry re-renders, re-evaluating the lookup — until the COSI Secret exists and the credentials materialise. This is the "bounded requeue re-render" option from #3247, implemented with machinery already in place; no controller changes, no new images, and it works on the currently pinned Flux 2.8.x.

Side effects, both intentional:

  • A bucket whose credentials are not usable yet reports not-Ready (with an explicit message naming the missing Secret) instead of Ready-with-missing-credentials.
  • A COSI Secret that exists but carries no BucketInfo key fails with a clear message instead of a cryptic b64dec nil error.

Why not the other #3247 options: an event-driven projection in cozystack-controller needs a watch on objectstorage.k8s.io CRDs that don't exist yet when the controller starts on fresh bootstraps (plus cache/RBAC surface); replacing the lookup with valuesFrom + targetPath cannot carry the BucketInfo JSON through helm-controller's --set parsing — literal: true fixes that but only exists since helm-controller 1.6 (Flux 2.9), while the platform pins Flux 2.8.x. This chart-only fix can later be superseded by the valuesFrom literal approach after a Flux bump.

Tests

  • helm-unittest suite for the gate (kubernetesProvider-mocked lookup): renders one Secret per user with trimmed endpoint once the COSI Secrets exist; fails with the retry message while one is missing; fails on a Secret without BucketInfo; renders nothing for a user-less bucket.
  • The bucket chainsaw e2e now asserts the child bucket-test-system release converges to Ready and both projected -credentials Secrets exist, and the S3 access check (mc) authenticates with keys taken from the projected Secrets instead of the raw COSI BucketInfo — proving end-to-end that the user-facing Secrets carry working credentials.

make -C packages/system/bucket test passes. helm lint failure on _namespace.host is pre-existing on main (chart has no defaults for the platform-injected _namespace).

Downstream repositories

  • No downstream repository is affected by this change

Release note

fix(bucket): the human-friendly `<bucket>-<user>-credentials` Secret is now created reliably even when the COSI driver provisions the bucket after the release's first reconcile. The bucket release reports not-Ready (and is retried automatically) until the credentials exist, instead of settling Ready with the Secret silently missing. If you previously created a `<bucket>-<user>-credentials` Secret manually to work around #3247, delete it (or annotate it for Helm adoption with `meta.helm.sh/release-name=<bucket>-system`, `meta.helm.sh/release-namespace=<namespace>` and label `app.kubernetes.io/managed-by=Helm`) before upgrading, otherwise the release upgrade fails with a resource-ownership conflict.

🤖 Generated with Claude Code

…ecret

The -credentials Secret rendered by system/bucket was gated on a Helm
lookup of the COSI-provisioned BucketInfo Secret. COSI provisions that
Secret asynchronously, after the BucketAccess is granted, and Helm only
re-evaluates lookup during a real install/upgrade attempt — so when the
Secret appeared after the release's last render, the -credentials Secret
was silently skipped and never created (nothing triggers another
render). Consumers such as the platform backup flow then hang forever on
the missing Secret (CredentialsProjectionPending).

Fail the render instead of skipping: the parent HelmRelease already sets
install/upgrade remediation retries to -1, so helm-controller keeps
retrying (re-rendering) the release until the COSI Secret exists, at
which point the credentials materialise. A bucket whose credentials are
not usable yet now reports not-Ready instead of Ready, and a Secret that
exists but carries no BucketInfo key fails with a clear message instead
of a cryptic b64dec error.

The bucket chainsaw e2e now asserts the child release converges to Ready
and both projected -credentials Secrets exist, and the S3 access check
authenticates with keys taken from the projected Secrets rather than the
raw COSI BucketInfo, proving the user-facing Secrets carry working
credentials.

Fixes #3247

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 90429107-49b3-4e17-9b58-3572da5666c0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/3247-bucket-credentials-lookup

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added size/L This PR changes 100-499 lines, ignoring generated files area/storage Issues or PRs related to storage (linstor, seaweedfs, bucket, velero, harbor) kind/bug Categorizes issue or PR as related to a bug labels Jul 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/storage Issues or PRs related to storage (linstor, seaweedfs, bucket, velero, harbor) kind/bug Categorizes issue or PR as related to a bug size/L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(system/bucket): credentials Secret never created when COSI secret appears after first reconcile (lookup-timing)

1 participant