Skip to content

cso: crash .... #337

Description

@garloff

/kind bug

cso crashes (-> CrashLoopBackoff)

What steps did you take and what happened:

A kind (0.31.0) cluster with capi-v1.12.5, capo-v0.14.2, cso: ghcr.io/sovereigncloudstack/cso:v0.2.0-alpha.1

Here's the pod log

W0528 17:52:20.625179       1 warnings.go:70] cluster.x-k8s.io/v1beta1 Cluster is deprecated; use cluster.x-k8s.io/v1beta2 Cluster
{"level":"INFO","time":"2026-05-28T17:52:20.720Z","file":"controller/controller.go:215","message":"Starting workers","controller":"cluster","controllerGroup":"cluster.x-k8s.io","controllerKind":"Cluster","worker count":1}
{"level":"INFO","time":"2026-05-28T17:52:20.721Z","file":"controller/controller.go:215","message":"Starting workers","controller":"clusteraddon","controllerGroup":"clusterstack.x-k8s.io","controllerKind":"ClusterAddon","worker count":1}
{"level":"INFO","time":"2026-05-28T17:52:20.721Z","file":"controller/controller.go:215","message":"Starting workers","controller":"clusterstackrelease","controllerGroup":"clusterstack.x-k8s.io","controllerKind":"ClusterStackRelease","worker count":1}
{"level":"INFO","time":"2026-05-28T17:52:20.721Z","file":"controller/controller.go:215","message":"Starting workers","controller":"clusterstack","controllerGroup":"clusterstack.x-k8s.io","controllerKind":"ClusterStack","worker count":1}
{"level":"INFO","time":"2026-05-28T17:52:30.863Z","file":"controller/controller.go:110","message":"Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference","controller":"clusteraddon","controllerGroup":"clusterstack.x-k8s.io","controllerKind":"ClusterAddon","ClusterAddon":{"name":"cluster-addon-cluster","namespace":"clusterns"},"namespace":"clusterns","name":"cluster-addon-cluster","reconcileID":"d5f1fc2e-97b6-4c41-9e0b-5ca08640aebc"}
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0xa8 pc=0x1b33c47]

goroutine 229 [running]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
        /src/cluster-stack-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:111 +0x1da
panic({0x1fb6760?, 0x3a2aac0?})
        /usr/local/go/src/runtime/panic.go:792 +0x132
github.com/SovereignCloudStack/cluster-stack-operator/pkg/release.(*Release).ClusterAddonValuesPath(...)
        /src/cluster-stack-operator/pkg/release/release.go:228
github.com/SovereignCloudStack/cluster-stack-operator/internal/controller.(*ClusterAddonReconciler).getOldReleaseObjects(0xc00018f200, {0x26c9bc0, 0xc0008cacc0}, 0xc0009ed958, {{0xc0007b2f00, 0x24}, {0xc0007b2f30, 0x2c}, 0xc00095b800}, 0x0)
        /src/cluster-stack-operator/internal/controller/clusteraddon_controller.go:566 +0x147
github.com/SovereignCloudStack/cluster-stack-operator/internal/controller.(*ClusterAddonReconciler).Reconcile(0xc00018f200, {0x26c9bc0, 0xc0008cacc0}, {{{0xc000450ab0?, 0x5?}, {0xc0000571a0?, 0xc000b49d10?}}})
        /src/cluster-stack-operator/internal/controller/clusteraddon_controller.go:473 +0x2a4f
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x26d0c78?, {0x26c9bc0?, 0xc0008cacc0?}, {{{0xc000450ab0?, 0xb?}, {0xc0000571a0?, 0x0?}}})
        /src/cluster-stack-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:114 +0xa5
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000422160, {0x26c9bf8, 0xc0002809b0}, {0x20f5a00, 0xc000420660})
        /src/cluster-stack-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:311 +0x39c
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000422160, {0x26c9bf8, 0xc0002809b0})
        /src/cluster-stack-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:261 +0x197
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
        /src/cluster-stack-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:222 +0x73
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2 in goroutine 124
        /src/cluster-stack-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:218 +0x469

What did you expect to happen:

Not crashing

Anything else you would like to add:

As I had patched csctl to also hash the cluster-class and accept to push a new release as soon as the hash of the cc has changed, the payload may surprise cso.

Environment:

  • cluster-stack-operator version: ghcr.io/sovereigncloudstack/cso@sha256:b82ef5b119d47670b3efd25c775744f5bbc5994f94a45c9898e05ddfe78811f0
  • Kubernetes version: (use kubectl version) 1.35.0 (kind)
  • OS (e.g. from /etc/os-release): openSUSE-15.6

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions