Skip to content

docs(skill): discover Kubernetes gateway workload #2262

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
elezar wants to merge 1 commit into
base: main
Choose a base branch
from
+34 −14
Draft

docs(skill): discover Kubernetes gateway workload #2262

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 34 additions & 14 deletions .agents/skills/debug-openshell-cluster/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,16 +132,22 @@ Common findings:
helm -n openshell status openshell
helm -n openshell get values openshell
kubectl -n openshell get deployment,statefulset,pod,svc,pvc
kubectl -n openshell logs deployment/openshell -c openshell-gateway --tail=200
kubectl -n openshell logs statefulset/openshell -c openshell-gateway --tail=200
kubectl -n openshell rollout status deployment/openshell
kubectl -n openshell rollout status statefulset/openshell
if kubectl -n openshell get deployment openshell >/dev/null 2>&1; then
GATEWAY_WORKLOAD="deployment/openshell"
elif kubectl -n openshell get statefulset openshell >/dev/null 2>&1; then
GATEWAY_WORKLOAD="statefulset/openshell"
else
echo "ERROR: neither deployment/openshell nor statefulset/openshell exists in namespace openshell" >&2
exit 1
fi
kubectl -n openshell logs "${GATEWAY_WORKLOAD}" -c openshell-gateway --tail=200
kubectl -n openshell rollout status "${GATEWAY_WORKLOAD}"
```

Use the log and rollout commands for the workload kind that exists in the
release. Look for failed installs, unexpected values, missing namespace, wrong
image tag, TLS settings that do not match the registered endpoint, and
scheduling failures.
The workload discovery fails explicitly if the release has neither supported
gateway controller. Look for failed installs, unexpected values, missing
namespace, wrong image tag, TLS settings that do not match the registered
endpoint, and scheduling failures.

For HA or PostgreSQL-backed installs, also check the external database Secret
referenced by `server.externalDbSecret` and the PostgreSQL workload if the test
Expand Down Expand Up @@ -196,11 +202,18 @@ label, supervisor env vars `OPENSHELL_K8S_SA_TOKEN_FILE` and
`OPENSHELL_PROVIDER_SPIFFE_WORKLOAD_API_SOCKET`, plus both the projected
`openshell-sa-token` volume and the `spiffe-workload-api` CSI volume.

Check the image references currently used by the gateway deployment:
Check the image references currently used by the gateway workload:

```bash
kubectl -n openshell get deployment openshell -o jsonpath="{.spec.template.spec.containers[*].image}{\"\n\"}{.spec.template.spec.containers[*].env[?(@.name==\"OPENSHELL_SUPERVISOR_IMAGE\")].value}{\"\n\"}"
kubectl -n openshell get statefulset openshell -o jsonpath="{.spec.template.spec.containers[*].image}{\"\n\"}{.spec.template.spec.containers[*].env[?(@.name==\"OPENSHELL_SUPERVISOR_IMAGE\")].value}{\"\n\"}"
if kubectl -n openshell get deployment openshell >/dev/null 2>&1; then
GATEWAY_WORKLOAD="deployment/openshell"
elif kubectl -n openshell get statefulset openshell >/dev/null 2>&1; then
GATEWAY_WORKLOAD="statefulset/openshell"
else
echo "ERROR: neither deployment/openshell nor statefulset/openshell exists in namespace openshell" >&2
exit 1
fi
kubectl -n openshell get "${GATEWAY_WORKLOAD}" -o jsonpath="{.spec.template.spec.containers[*].image}{\"\n\"}{.spec.template.spec.containers[*].env[?(@.name==\"OPENSHELL_SUPERVISOR_IMAGE\")].value}{\"\n\"}"
helm -n openshell get values openshell | grep -E 'repository|tag|supervisorImage|workload'
```

Expand Down Expand Up @@ -244,8 +257,15 @@ If the gateway is healthy but sandbox creation fails:
```bash
kubectl -n openshell get pods
kubectl -n openshell get events --sort-by=.lastTimestamp | tail -n 50
kubectl -n openshell logs deployment/openshell -c openshell-gateway --tail=200
kubectl -n openshell logs statefulset/openshell -c openshell-gateway --tail=200
if kubectl -n openshell get deployment openshell >/dev/null 2>&1; then
GATEWAY_WORKLOAD="deployment/openshell"
elif kubectl -n openshell get statefulset openshell >/dev/null 2>&1; then
GATEWAY_WORKLOAD="statefulset/openshell"
else
echo "ERROR: neither deployment/openshell nor statefulset/openshell exists in namespace openshell" >&2
exit 1
fi
kubectl -n openshell logs "${GATEWAY_WORKLOAD}" -c openshell-gateway --tail=200
```

Check the configured sandbox namespace:
Expand Down Expand Up @@ -344,7 +364,7 @@ openshell logs <sandbox-name>
| Docker or Podman sandbox never registers | Wrong callback endpoint or supervisor startup failure | Gateway logs and sandbox container logs |
| Docker GPU e2e fails before GPU sandbox comparison | NVIDIA CDI specs are missing or Docker has not discovered them | `docker info --format '{{json .DiscoveredDevices}}'`, `/etc/cdi`, `/var/run/cdi`, `nvidia-cdi-refresh.service` |
| Kubernetes gateway pod pending | PVC unbound, taint, selector, or insufficient resources | `kubectl -n openshell describe pod <pod>` |
| Kubernetes gateway pod crash loops | Missing secret, bad DB URL, bad TLS config | `kubectl -n openshell logs deployment/openshell -c openshell-gateway` or `kubectl -n openshell logs statefulset/openshell -c openshell-gateway` |
| Kubernetes gateway pod crash loops | Missing secret, bad DB URL, bad TLS config | Set `GATEWAY_WORKLOAD` as shown in Step 5, then run `kubectl -n openshell logs "${GATEWAY_WORKLOAD}" -c openshell-gateway` |
| CLI TLS error | Local mTLS bundle does not match server cert/CA | Check `~/.config/openshell/gateways/<name>/mtls/` |
| Image pull failure | Gateway or sandbox image cannot be pulled | Runtime events and image pull credentials |
| `K8s namespace not ready` with `envoy-gateway-openshell.yaml: the server could not find the requested resource` | Optional Gateway API manifest was applied without Envoy Gateway CRDs, or k3s Helm controller startup exceeded the namespace wait | Apply `deploy/kube/manifests/envoy-gateway-openshell.yaml` manually only after Envoy Gateway is installed and `grpcRoute` is enabled |
Expand Down
Loading