Skip to content

Commit 39d95b4

Browse files
authored
chore(e2e): remove obsolete OpenShift sandbox SCC grant (#4823)
Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
1 parent 0a51652 commit 39d95b4

3 files changed

Lines changed: 14 additions & 24 deletions

File tree

‎TESTING.md‎

Lines changed: 13 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -447,14 +447,19 @@ OPENSHELL_E2E_KUBE_TEST=smoke mise run e2e:kubernetes
447447
```
448448

449449
**OpenShift**: when the target cluster exposes the `route.openshift.io` API
450-
group, the harness automatically applies SCC-compatible Helm overrides, grants
451-
the required SCCs (`privileged` to `openshell-sandbox`, and `anyuid` to the
452-
PostgreSQL fixture for DB scenarios), and drives the gateway through a
453-
passthrough Route with mandatory mTLS instead of port-forward. No extra flags are
454-
needed, but `oc` must be installed and authenticated against the target cluster
455-
with permission to modify SCC bindings (`oc adm policy add-scc-to-user`) — the
456-
harness exits early if `oc` is missing. The SCC grants and extracted client
457-
mTLS material are removed during cleanup, including on failure or interrupt.
450+
group, the harness automatically applies SCC-compatible Helm overrides and
451+
drives the gateway through a passthrough Route with mandatory mTLS instead of
452+
port-forward. No extra flags are needed, but `oc` must be installed and
453+
authenticated against the target cluster; the harness exits early if `oc` is
454+
missing. DB scenarios grant `anyuid` to the PostgreSQL fixture and require
455+
permission to modify SCC bindings (`oc adm policy add-scc-to-user`). That grant
456+
and extracted client mTLS material are removed during cleanup, including on
457+
failure or interrupt.
458+
459+
Sandbox pods use the namespace-assigned UID/GID and run under `restricted-v2`
460+
without a privileged SCC grant to `openshell-sandbox`. Earlier versions required
461+
that grant for in-container isolation setup. [RFC 0012](https://github.com/NVIDIA/OpenShell/blob/main/rfc/0012-isolation-backend/README.md#motivation)
462+
explains the isolation backend separation behind the capability-free runtime.
458463

459464
On a **remote** cluster, drop the `e2e-host-gateway` feature. Those tests rely
460465
on the sandbox-side `host.openshell.internal` alias reaching the machine running

‎e2e/with-kube-gateway.sh‎

Lines changed: 0 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -119,7 +119,6 @@ CORPORATE_PROXY_FIXTURE_SECRET="openshell-e2e-proxy-auth"
119119
CORPORATE_PROXY_FIXTURE_CA_CONFIGMAP="openshell-e2e-proxy-ca"
120120
CORPORATE_PROXY_CA_FIXTURE_DEPLOYED=0
121121
OPENSHIFT_DETECTED=0
122-
OPENSHIFT_SANDBOX_SCC_GRANTED=0
123122
OPENSHIFT_POSTGRES_SCC_GRANTED=0
124123
OPENSHIFT_ROUTE_HOST=""
125124
# Temp dir holding the client mTLS material extracted from openshell-client-tls
@@ -589,14 +588,6 @@ cleanup() {
589588
--ignore-not-found >/dev/null 2>&1 || true
590589
fi
591590

592-
if [ "${OPENSHIFT_SANDBOX_SCC_GRANTED}" = "1" ]; then
593-
oc adm policy remove-scc-from-user privileged \
594-
--context "${KUBE_CONTEXT}" \
595-
-z openshell-sandbox -n "${NAMESPACE}" \
596-
2>/dev/null || true
597-
OPENSHIFT_SANDBOX_SCC_GRANTED=0
598-
fi
599-
600591
# Remove the extracted client mTLS material (also covered by the WORKDIR sweep
601592
# below, but drop the private key promptly and explicitly).
602593
if [ -n "${OPENSHIFT_PKI_DIR}" ]; then
@@ -1259,12 +1250,6 @@ if [ "${OPENSHIFT_DETECTED}" = "1" ]; then
12591250

12601251
kctl create namespace "${NAMESPACE}" --dry-run=client -o yaml | kctl apply -f -
12611252

1262-
echo "Granting privileged SCC to openshell-sandbox in namespace ${NAMESPACE}..."
1263-
oc adm policy add-scc-to-user privileged \
1264-
--context "${KUBE_CONTEXT}" \
1265-
-z openshell-sandbox -n "${NAMESPACE}"
1266-
OPENSHIFT_SANDBOX_SCC_GRANTED=1
1267-
12681253
# Drive the gateway through a passthrough Route with mTLS instead of
12691254
# port-forward. The Route host is deterministic: OpenShift serves any name
12701255
# under the cluster ingress (apps) domain via the router's wildcard, so we

‎skills/debug-openshell-cluster/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1280,7 +1280,7 @@ credential failures.
12801280
| Gateway or `<release>-certgen` hook pod rejected with `violates PodSecurity "restricted:latest"` | The rendered container lacks `runAsNonRoot` or `seccompProfile`: an older chart version does not set them, or an override in `securityContext` / `pkiInitJob.securityContext` unset them | `kubectl -n openshell get events --field-selector reason=FailedCreate`; upgrade the chart, then remove overrides that unset those fields |
12811281
| OpenShift gateway pod fails to start with an SCC/`runAsUser` error (e.g. `unable to validate against any security context constraint`) | Chart's default `podSecurityContext`/`securityContext` hardcodes `runAsUser`/`fsGroup`, which the restricted-v2 SCC rejects; it must instead inject the namespace-assigned UID/GID range | `oc -n openshell describe pod <pod>`; deploy with `podSecurityContext: null` and clear `securityContext.runAsUser` (see `deploy/helm/openshell/ci/values-openshift-scc.yaml`) |
12821282
| OpenShift `<release>-supervisor` Deployment never creates pods after setting `supervisor_topology = "gateway"` (SCC `runAsUser`/`fsGroup` not in the namespace range) | `supervisor.podSecurityContext` defaults `runAsUser`, `runAsGroup`, and `fsGroup` to 65534, outside the namespace's assigned UID/GID ranges | `kubectl -n openshell get events --field-selector reason=FailedCreate`; compare with the `openshift.io/sa.scc.*` annotations from `oc describe ns openshell`; set `supervisor.podSecurityContext.runAsUser`, `.runAsGroup`, and `.fsGroup` to `null`, as `deploy/helm/openshell/ci/values-openshift-scc.yaml` does, and keep `runAsNonRoot` and `seccompProfile` |
1283-
| OpenShift sandbox pod fails to start (`unable to validate against any security context constraint`) | The `openshell-sandbox` service account lacks the privileged SCC it needs | `oc adm policy add-scc-to-user privileged -z openshell-sandbox -n openshell`; remove with `remove-scc-from-user` when done |
1283+
| OpenShift sandbox pod fails to start (`unable to validate against any security context constraint`) | An explicit sandbox UID/GID override or custom security context conflicts with the namespace's SCC range | Inspect pod events and the namespace's `openshift.io/sa.scc.uid-range` and `openshift.io/sa.scc.supplemental-groups` annotations; remove incompatible `sandbox_uid` / `sandbox_gid` overrides so the driver uses the namespace-assigned identity. Current sandbox pods use `restricted-v2` without a privileged SCC grant; see the [OpenShift guide](https://docs.nvidia.com/openshell/latest/kubernetes/openshift) and [RFC 0012](https://github.com/NVIDIA/OpenShell/blob/main/rfc/0012-isolation-backend/README.md#motivation). |
12841284
| OpenShift self-hosted Vault/OpenBao credential store pod never schedules (waits time out with `no matching resources found`) | The store's Helm chart pins `runAsUser`/`fsGroup`/seccomp, which restricted-v2 rejects, so the StatefulSet controller never creates the pod | Deploy the store's chart in its OpenShift mode (`--set global.openshift=true` for the OpenBao/Vault chart) so the namespace SCC assigns a compliant security context — no manual SCC grant needed |
12851285
| Vault credential driver returns HTTP 403 / `Vault Kubernetes auth denied the configured role` on provider create | Vault's `auth/kubernetes` method or the gateway login role is not provisioned, or the role is not bound to the gateway service account and namespace | In Vault: `bao auth enable kubernetes` and `bao write auth/kubernetes/config kubernetes_host=... kubernetes_ca_cert=@...`; ensure the login role's `bound_service_account_names`/`bound_service_account_namespaces` match the gateway SA and namespace and its policy grants the credential paths |
12861286
| CLI TLS error | Local mTLS bundle does not match server cert/CA | Check `~/.config/openshell/gateways/<name>/mtls/` |

0 commit comments

Comments
 (0)