docs(kubernetes): clarify Agent Sandbox extension CRD prerequisite

Signed-off-by: Gordon Sim <gsim@redhat.com>

Author: grs

.agents/skills/debug-openshell-cluster/SKILL.md

@@ -572,6 +572,7 @@ openshell logs <sandbox-name>
 | Kubernetes gateway pod pending | PVC unbound, taint, selector, or insufficient resources | `kubectl -n openshell describe pod <pod>` |
 | Kubernetes sandbox pod stuck pending, workspace PVC unbound | Cluster has no default `StorageClass` and OpenShell does not set `storageClassName` on the workspace PVC (clusters with a default `StorageClass` bind fine without it) | `kubectl -n openshell describe pvc`; set `server.workspaceStorageClass` (gateway config `workspace_storage_class`) to a valid `StorageClass` |
 | Kubernetes gateway pod crash loops | Missing secret, bad DB URL, bad TLS config | `kubectl -n openshell logs deployment/openshell -c openshell-gateway` or `kubectl -n openshell logs statefulset/openshell -c openshell-gateway` |
+| Gateway logs `Agent Sandbox extension APIs are unavailable` and Kubernetes creates cold sandboxes | Agent Sandbox was installed with `manifest.yaml` only while OpenShell warm pooling defaults are enabled | Apply Agent Sandbox `extensions.yaml`, or set `server.warmPooling.enabled=false`; then restart the gateway so the driver revalidates `SandboxClaim`, `SandboxTemplate`, and `SandboxWarmPool` APIs |
 | CLI TLS error | Local mTLS bundle does not match server cert/CA | Check `~/.config/openshell/gateways/<name>/mtls/` |
 | Edge or OIDC gateway returns `Unauthenticated` | Stored login expired, audience/scopes mismatch, or gateway auth configuration changed | `openshell gateway info`, `openshell gateway login <name>`, gateway auth logs |
 | Gateway fails before serving health after enabling an interceptor | Interceptor endpoint unavailable or manifest/binding validation failed | Gateway and interceptor logs; interceptor socket; `binding_policy`, phases, and failure policy |

crates/openshell-driver-kubernetes/src/driver.rs

@@ -618,6 +618,7 @@ pub struct KubernetesComputeDriver {
     watch_client: Client,
     sandbox_api_version: Arc<OnceCell<&'static str>>,
     warm_pool_cache: WarmPoolCache,
+    warm_pool_extension_apis_available: bool,
     config: KubernetesComputeConfig,
     operator_allowlist: Option<OperatorNamespaceAllowlist>,
 }
@@ -699,15 +700,42 @@ impl KubernetesComputeDriver {
             watch_client,
             sandbox_api_version: Arc::new(OnceCell::new()),
             warm_pool_cache: WarmPoolCache::default(),
+            warm_pool_extension_apis_available: false,
             config,
             operator_allowlist,
         };
+        let warm_pool_extension_apis_available = if driver.config.warm_pooling.enabled {
+            match driver.validate_warm_pool_extension_apis().await {
+                Ok(true) => true,
+                Ok(false) => {
+                    warn!(
+                        namespace = %driver.config.namespace,
+                        "Agent Sandbox extension APIs are unavailable; Kubernetes warm pooling and template reconciliation are disabled. Install Agent Sandbox extensions.yaml or disable warm_pooling."
+                    );
+                    false
+                }
+                Err(err) => {
+                    warn!(
+                        namespace = %driver.config.namespace,
+                        error = %err,
+                        "Could not validate Agent Sandbox extension APIs; Kubernetes warm pooling and template reconciliation are disabled"
+                    );
+                    false
+                }
+            }
+        } else {
+            false
+        };
+        let driver = Self {
+            warm_pool_extension_apis_available,
+            ..driver
+        };
 
         if driver.workspace_mode() == WorkspaceMode::Shared {
             driver.backfill_gateway_id_labels().await?;
         }
 
-        if driver.config.warm_pooling.enabled {
+        if driver.config.warm_pooling.enabled && driver.warm_pool_extension_apis_available {
             driver.spawn_warm_pool_cache_controller();
         }
         Ok(driver)
@@ -719,7 +747,9 @@ impl KubernetesComputeDriver {
                 "kubernetes",
                 openshell_core::VERSION,
                 &self.config.default_image,
-                self.config.warm_pooling.enabled && self.config.warm_pooling.templates.enabled,
+                self.config.warm_pooling.enabled
+                    && self.config.warm_pooling.templates.enabled
+                    && self.warm_pool_extension_apis_available,
             ),
         )
     }
@@ -1175,6 +1205,15 @@ impl KubernetesComputeDriver {
         &self,
         template: &DriverSandboxTemplateResource,
     ) -> Result<(), KubernetesDriverError> {
+        if !self.warm_pool_extension_apis_available {
+            debug!(
+                template_id = %template.id,
+                template_name = %template.name,
+                workspace = %template.workspace,
+                "Skipping sandbox template warm-pool reconciliation because Agent Sandbox extension APIs are unavailable"
+            );
+            return Ok(());
+        }
         let maybe_rendered = render_warm_pool_template(self.client.clone(), &self.config, template)
             .await
             .map_err(KubernetesDriverError::InvalidArgument)?;
@@ -1211,6 +1250,14 @@ impl KubernetesComputeDriver {
         template_id: &str,
         workspace: &str,
     ) -> Result<(), KubernetesDriverError> {
+        if !self.warm_pool_extension_apis_available {
+            debug!(
+                template_id,
+                workspace,
+                "Skipping sandbox template warm-pool cleanup because Agent Sandbox extension APIs are unavailable"
+            );
+            return Ok(());
+        }
         garbage_collect_warm_pool_template(
             self.client.clone(),
             &self.config,
@@ -1316,6 +1363,68 @@ impl KubernetesComputeDriver {
         Self::namespaced_extension_api(client, namespace, SANDBOX_TEMPLATE_KIND)
     }
 
+    async fn validate_warm_pool_extension_apis(&self) -> Result<bool, String> {
+        for kind in [
+            SANDBOX_CLAIM_KIND,
+            SANDBOX_TEMPLATE_KIND,
+            SANDBOX_WARM_POOL_KIND,
+        ] {
+            if !Self::extension_api_available(self.client.clone(), &self.config.namespace, kind)
+                .await?
+            {
+                debug!(
+                    namespace = %self.config.namespace,
+                    kind,
+                    api_group = EXTENSIONS_GROUP,
+                    api_version = EXTENSIONS_VERSION_V1BETA1,
+                    "Agent Sandbox extension API is unavailable"
+                );
+                return Ok(false);
+            }
+        }
+        Ok(true)
+    }
+
+    async fn extension_api_available(
+        client: Client,
+        fallback_namespace: &str,
+        kind: &str,
+    ) -> Result<bool, String> {
+        let all_api = Self::all_extension_api(client.clone(), kind);
+        match tokio::time::timeout(
+            KUBE_API_TIMEOUT,
+            all_api.api.list(&ListParams::default().limit(1)),
+        )
+        .await
+        {
+            Ok(Ok(_)) => Ok(true),
+            Ok(Err(err)) if extension_api_unavailable(&err) => Ok(false),
+            Ok(Err(KubeError::Api(err))) if err.code == 403 => {
+                let namespaced_api =
+                    Self::namespaced_extension_api(client, fallback_namespace, kind);
+                match tokio::time::timeout(
+                    KUBE_API_TIMEOUT,
+                    namespaced_api.api.list(&ListParams::default().limit(1)),
+                )
+                .await
+                {
+                    Ok(Ok(_)) => Ok(true),
+                    Ok(Err(err)) if extension_api_unavailable(&err) => Ok(false),
+                    Ok(Err(err)) => Err(err.to_string()),
+                    Err(_) => Err(format!(
+                        "timed out after {}s waiting for Kubernetes API",
+                        KUBE_API_TIMEOUT.as_secs()
+                    )),
+                }
+            }
+            Ok(Err(err)) => Err(err.to_string()),
+            Err(_) => Err(format!(
+                "timed out after {}s waiting for Kubernetes API",
+                KUBE_API_TIMEOUT.as_secs()
+            )),
+        }
+    }
+
     fn spawn_warm_pool_cache_controller(&self) {
         let cache = self.warm_pool_cache.clone();
         let client = self.client.clone();
@@ -1833,7 +1942,7 @@ impl KubernetesComputeDriver {
         sandbox_template: &DriverSandboxTemplateRef,
         request_fingerprint: &str,
     ) -> Result<bool, KubernetesDriverError> {
-        if !self.config.warm_pooling.enabled {
+        if !self.config.warm_pooling.enabled || !self.warm_pool_extension_apis_available {
             return Ok(false);
         }
 
@@ -2183,6 +2292,7 @@ impl KubernetesComputeDriver {
         let data = sandbox_to_k8s_spec(sandbox.spec.as_ref(), &params)
             .map_err(KubernetesDriverError::InvalidArgument)?;
         if self.config.warm_pooling.enabled
+            && self.warm_pool_extension_apis_available
             && let Some(sandbox_template) = sandbox_template
         {
             match sandbox_spec_fingerprint(&data) {
@@ -2572,47 +2682,48 @@ impl KubernetesComputeDriver {
         let mut sandbox_stream = watcher::watcher(agent_sandbox_api.api, watcher_config).boxed();
         let claim_watcher_config = watcher::Config::default()
             .labels(&format!("{LABEL_MANAGED_BY}={LABEL_MANAGED_BY_VALUE}"));
-        let claim_api = if self.config.warm_pooling.enabled {
-            let all_api = Self::sandbox_claim_api_all(self.watch_client.clone());
-            match tokio::time::timeout(
-                KUBE_API_TIMEOUT,
-                all_api.api.list(&ListParams::default().limit(1)),
-            )
-            .await
-            {
-                Ok(Ok(_)) => Some(all_api.api),
-                Ok(Err(KubeError::Api(err))) if err.code == 403 => {
-                    let namespaced_api =
-                        Self::sandbox_claim_api(self.watch_client.clone(), &namespace);
-                    match tokio::time::timeout(
-                        KUBE_API_TIMEOUT,
-                        namespaced_api.api.list(&ListParams::default().limit(1)),
-                    )
-                    .await
-                    {
-                        Ok(Ok(_)) => Some(namespaced_api.api),
-                        Ok(Err(err)) if extension_api_unavailable(&err) => None,
-                        Ok(Err(err)) => return Err(err.to_string()),
-                        Err(_) => {
-                            return Err(format!(
-                                "timed out after {}s waiting for Kubernetes API",
-                                KUBE_API_TIMEOUT.as_secs()
-                            ));
+        let claim_api =
+            if self.config.warm_pooling.enabled && self.warm_pool_extension_apis_available {
+                let all_api = Self::sandbox_claim_api_all(self.watch_client.clone());
+                match tokio::time::timeout(
+                    KUBE_API_TIMEOUT,
+                    all_api.api.list(&ListParams::default().limit(1)),
+                )
+                .await
+                {
+                    Ok(Ok(_)) => Some(all_api.api),
+                    Ok(Err(KubeError::Api(err))) if err.code == 403 => {
+                        let namespaced_api =
+                            Self::sandbox_claim_api(self.watch_client.clone(), &namespace);
+                        match tokio::time::timeout(
+                            KUBE_API_TIMEOUT,
+                            namespaced_api.api.list(&ListParams::default().limit(1)),
+                        )
+                        .await
+                        {
+                            Ok(Ok(_)) => Some(namespaced_api.api),
+                            Ok(Err(err)) if extension_api_unavailable(&err) => None,
+                            Ok(Err(err)) => return Err(err.to_string()),
+                            Err(_) => {
+                                return Err(format!(
+                                    "timed out after {}s waiting for Kubernetes API",
+                                    KUBE_API_TIMEOUT.as_secs()
+                                ));
+                            }
                         }
                     }
+                    Ok(Err(err)) if extension_api_unavailable(&err) => None,
+                    Ok(Err(err)) => return Err(err.to_string()),
+                    Err(_) => {
+                        return Err(format!(
+                            "timed out after {}s waiting for Kubernetes API",
+                            KUBE_API_TIMEOUT.as_secs()
+                        ));
+                    }
                 }
-                Ok(Err(err)) if extension_api_unavailable(&err) => None,
-                Ok(Err(err)) => return Err(err.to_string()),
-                Err(_) => {
-                    return Err(format!(
-                        "timed out after {}s waiting for Kubernetes API",
-                        KUBE_API_TIMEOUT.as_secs()
-                    ));
-                }
-            }
-        } else {
-            None
-        };
+            } else {
+                None
+            };
         let mut claim_stream = claim_api.map_or_else(
             || stream::pending().boxed(),
             |api| watcher::watcher(api, claim_watcher_config).boxed(),

deploy/helm/openshell/README.md

@@ -17,6 +17,12 @@ The Kubernetes Agent Sandbox CRDs and controller must be installed on the cluste
 kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/latest/download/manifest.yaml
 ```
 
+The chart defaults Kubernetes warm pooling on. Install the Agent Sandbox extension APIs too, or set `server.warmPooling.enabled=false`:
+
+```shell
+kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/latest/download/extensions.yaml
+```
+
 ## Install on Kubernetes
 
 ```shell

deploy/helm/openshell/README.md.gotmpl

@@ -17,6 +17,12 @@ The Kubernetes Agent Sandbox CRDs and controller must be installed on the cluste
 kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/latest/download/manifest.yaml
 ```
 
+The chart defaults Kubernetes warm pooling on. Install the Agent Sandbox extension APIs too, or set `server.warmPooling.enabled=false`:
+
+```shell
+kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/latest/download/extensions.yaml
+```
+
 ## Install on Kubernetes
 
 ```shell

docs/kubernetes/setup.mdx

@@ -22,7 +22,7 @@ Make sure the following are in place before you install.
 |---|---|---|
 | Kubernetes 1.29+ with RBAC enabled | Yes | No additional notes. |
 | Helm 3.x | Yes | No additional notes. |
-| Agent Sandbox controller and CRDs | Yes | Install before the OpenShell chart. Refer to [Install Agent Sandbox](#install-agent-sandbox). |
+| Agent Sandbox controller and CRDs | Yes | Install before the OpenShell chart. Refer to [Install Agent Sandbox](#install-agent-sandbox). Install `extensions.yaml` too when warm pooling is enabled. |
 | cert-manager | No | Refer to [Managing Certificates](/kubernetes/managing-certificates). Use cert-manager only if you prefer it over the built-in PKI job. |
 | Kubernetes Gateway API | No | Refer to [Ingress](/kubernetes/ingress). Use it only for external access without port-forwarding. |
 
@@ -38,6 +38,14 @@ kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/lates
 
 This creates the `agent-sandbox-system` namespace, installs the `sandboxes.agents.x-k8s.io` CRD, and starts the controller.
 
+Install the Agent Sandbox extension APIs when Kubernetes warm pooling is enabled. The OpenShell chart defaults warm pooling on, but it does not install the `SandboxClaim`, `SandboxTemplate`, or `SandboxWarmPool` CRDs:
+
+```shell
+kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/latest/download/extensions.yaml
+```
+
+If you do not install `extensions.yaml`, set `server.warmPooling.enabled=false`. Otherwise OpenShell disables warm pooling and template reconciliation at startup and falls back to direct `Sandbox` creation.
+
 <Note>
 **Air-gapped clusters:** mirror the manifest above and the `registry.k8s.io/agent-sandbox/agent-sandbox-controller` image referenced inside it to your internal registry, then point the manifest's image reference at your mirror before applying. You will also need to mirror the OpenShell gateway and sandbox images — see the chart's `image.repository` value for the gateway and `server.sandboxImage` / `server.supervisorImage` for the sandbox runtime.
 </Note>
@@ -258,7 +266,7 @@ when `server.warmPooling.enabled=true`:
 | `""` | `nodes` | get, list, watch |
 | `""` | `namespaces` | get |
 | `extensions.agents.x-k8s.io` | `sandboxclaims` | get, list, watch; create, delete when warm pooling is enabled |
-| `extensions.agents.x-k8s.io` | `sandboxtemplates`, `sandboxwarmpools` | get, list, watch; create, patch, update, delete when profile reconciliation is enabled |
+| `extensions.agents.x-k8s.io` | `sandboxtemplates`, `sandboxwarmpools` | get, list, watch; create, patch, update, delete when template reconciliation is enabled |
 
 To use an existing ServiceAccount instead of creating one, set `serviceAccount.create=false` and supply its name:
 

docs/reference/gateway-config.mdx

@@ -536,6 +536,9 @@ gateway_pod_selector = { "app.kubernetes.io/name" = "openshell", "app.kubernetes
 [openshell.drivers.kubernetes.warm_pooling]
 # When enabled, compatible creates may use v1beta1 SandboxClaim resources
 # backed by the driver's cache of OpenShell-enabled SandboxWarmPool resources.
+# Requires the Agent Sandbox extensions.yaml APIs. If they are unavailable at
+# gateway startup, OpenShell disables warm pooling and template reconciliation
+# for this driver instance and falls back to direct Sandbox creation.
 enabled = true
 
 [openshell.drivers.kubernetes.warm_pooling.templates]

docs/reference/sandbox-compute-drivers.mdx

@@ -462,6 +462,13 @@ OpenShell workspace. The current shared-namespace mapping selects the configured
 `openshell.drivers.kubernetes.warm_pooling.enabled = false` to force direct
 `Sandbox` creation.
 
+Warm pooling requires the Agent Sandbox extension APIs from `extensions.yaml`.
+The OpenShell chart grants RBAC for those APIs, but it does not install the
+`SandboxClaim`, `SandboxTemplate`, or `SandboxWarmPool` CRDs. If the
+extension APIs are unavailable when the Kubernetes driver starts, OpenShell
+disables warm pooling and template reconciliation for that driver instance and
+continues with direct `Sandbox` creation.
+
 When `openshell.drivers.kubernetes.warm_pooling.templates.enabled = true`, the
 gateway notifies the Kubernetes driver when OpenShell `SandboxTemplate`
 resources are created, deleted, or synced at gateway startup. The driver