Put the node's IAM role out of a Bot's reach in the EKS recipe (#620)

* Put the node's IAM role out of a Bot's reach in the EKS recipe

A Bot's computer runs a shell, and a shell reaches whatever its pod
reaches. On EKS that includes 169.254.169.254, which hands out the node's
IAM role, and it is one `run_command` away: OpenBot's policy engine
decides browser navigation and tool calls, and the shell reaches the
network directly.

The NetworkPolicy already excepts `169.254.0.0/16` and is the better
control where it is enforced. It is off by default, for the reason the
chart README gives, and on EKS it does nothing until the VPC CNI is told
to enforce it, so the exception is not what holds in the configuration a
cluster actually starts in.

The two node-group lines are one control rather than two.
`disablePodIMDS` sets the hop limit to 1, which governs the IMDSv2 token
PUT; IMDSv1 is a bare GET with no PUT to limit, so the hop limit stops
nothing until `disableIMDSv1` makes a token mandatory. Written out
because the first alone does not work. Covers pods on the cluster
network; `hostNetwork: true` is on the node and is not covered.

Nothing in OpenBot wants pod-level IMDS: the chart reaches AWS through
IRSA, a ServiceAccount annotation and a projected token.

Documentation only. No chart template changes, and an existing cluster is
unaffected until its node group is recreated.

* Say what disableIMDSv1 actually does here, and what the hop limit costs

The text had `disablePodIMDS` doing nothing until `disableIMDSv1` made a token
mandatory. eksctl's own documentation says otherwise: `disableIMDSv1` defaults
to true, and either flag sets `httpTokens: required`. So `disablePodIMDS` is the
line that does the work and the other one states a default rather than changing
it, which is worth writing out but not worth describing as the thing that makes
the first one work.

It also gave the control with no cost attached. A hop limit of 1 is exactly what
the EBS CSI driver asks you not to set: its FAQ wants 2 or greater in a
containerized environment, and `MutableCSINodeAllocatableCount` requires IMDS to
be the driver's metadata source. Somebody following this recipe on a cluster
using either should know before their volumes do.

---------

Co-authored-by: David McKay <david@copilotkit.ai>

Author: CamoRageaholic1

CHANGELOG.md

@@ -156,6 +156,28 @@ a deployment.
 `docs/architecture.md`, `docs/deployment.md` and `docs/configuration.md` all described the old rule
 and now describe this one.
 
+### The EKS cluster recipe puts the node's IAM role out of a Bot's reach
+
+A Bot has a shell, and a shell reaches whatever its pod reaches, including the instance metadata
+service at 169.254.169.254 that hands out the node's IAM role. The NetworkPolicy excepts that
+address, but it is off by default and does nothing on EKS until the VPC CNI is told to enforce it,
+so the cluster config in the chart README now sets `disableIMDSv1` and `disablePodIMDS` on the node
+group, which closes it at the node whatever the CNI is doing.
+
+`disablePodIMDS` is the line that does the work: it sets the hop limit to 1, and that governs the
+IMDSv2 token PUT, so a pod one hop further out than the node cannot get a token. `disableIMDSv1` is
+written beside it to say so out loud rather than to change anything, because eksctl defaults it to
+true and sets `httpTokens: required` for either flag. Pods on the cluster network are covered; one
+running with `hostNetwork: true` is on the node and is not.
+
+**It costs something, and the recipe says so.** The EBS CSI driver reads instance metadata from
+IMDS, and its own documentation asks for a hop limit of 2 or greater in a containerized environment.
+A hop limit of 1 also rules out `MutableCSINodeAllocatableCount`, which requires IMDS to be the
+driver's metadata source. Take these two lines out if you need that.
+
+Nothing in OpenBot wants pod-level IMDS: the chart reaches AWS through IRSA. Documentation only; no
+chart template changed, and an existing cluster is unaffected until its node group is recreated.
+
 ### A coworker can be a file of its own
 
 The example package declared every coworker in one `agents.yaml`, so adding one meant editing a file

charts/openbot/README.md

@@ -82,8 +82,44 @@ managedNodeGroups:
     maxSize: 4
     volumeSize: 60
     volumeType: gp3
+    # A Bot has a shell, and a shell reaches whatever its pod reaches. These two put the node's
+    # own IAM role out of that reach. See below.
+    disableIMDSv1: true
+    disablePodIMDS: true
 ```
 
+**Why the two IMDS lines.** A node's instance metadata service, at 169.254.169.254, hands out that
+node's IAM role to anything that asks from the node. A Bot's computer is a pod on that node running
+commands a model chose, so `curl` against that address is one `run_command` away, and no rule in
+OpenBot's own policy engine is consulted: the engine decides browser navigation and tool calls, and
+the shell reaches the network directly.
+
+`disablePodIMDS` sets the hop limit to 1, so a packet from a pod, which has one more hop to make
+than one from the node itself, no longer arrives. That hop limit applies to the IMDSv2 token PUT,
+and IMDSv1 is a bare GET with no PUT to limit, so the hop limit only means anything once a token is
+mandatory. It already is: eksctl documents `disableIMDSv1` as defaulting to true, and sets
+`httpTokens: required` for either flag. `disableIMDSv1: true` is written out here to state that
+rather than to change it, since a reader deciding whether this recipe is safe should not have to
+know an upstream default. Note this covers pods on the cluster network; a pod running with
+`hostNetwork: true` is on the node and still reaches the address.
+
+**What the hop limit costs.** The EBS CSI driver reads EC2 instance metadata from IMDS, and its own
+FAQ asks for "the hop limit for IMDSv2 responses ... set to 2 or greater" in a containerized
+environment like EKS. A hop limit of 1 also rules out `MutableCSINodeAllocatableCount`, whose
+documented prerequisites are that the driver "must be using IMDS Metadata" and that IMDS be its only
+enabled or preferred metadata source. Drop these two lines if your cluster needs either, and lean on
+the NetworkPolicy below instead.
+
+Two things eksctl will refuse rather than warn about, both worth knowing before the next edit to
+this file: `disablePodIMDS` cannot be combined with `iam.withAddonPolicies` on the same node group,
+and neither field may be set on a node group that supplies its own `launchTemplate`.
+
+The NetworkPolicy below also excepts `169.254.0.0/16`, and is the better control where it is
+enforced. It is off by default and does nothing on EKS until the VPC CNI is told to enforce it, so
+these two lines are what holds in the configuration a cluster actually starts in. Nothing in OpenBot
+wants pod-level IMDS either way: the chart reaches AWS through IRSA, which is a ServiceAccount
+annotation and a projected token rather than the node's role.
+
 ```sh
 eksctl create cluster -f cluster.yaml
 
@@ -401,7 +437,9 @@ reach. If your release has `networkPolicy.enabled` and `computers.mode: sandbox`
 set the range and run it again, and the policy is narrow for the first time.
 
 A Bot's computer is allowed 80 and 443 to public addresses and nothing else, which is what stops a
-browser reaching the cluster, the database, or the cloud's credential endpoint. A per-Bot egress
+browser reaching the cluster, the database, or the cloud's credential endpoint. That last one is
+worth closing at the node as well, because this policy is off until somebody turns it on: see
+`disablePodIMDS` in the EKS cluster config above. A per-Bot egress
 proxy is therefore two settings rather than one: the variable that names it, and the rule that lets
 the computer reach it.