terraform-aws-eks-gpu-nodepool

An end-to-end Terraform stack for running GPU workloads on EKS: VPC, EKS control plane, and a GPU-backed managed node group -- correctly tainted, sized for spot or on-demand, and wired up with the IAM permissions an EKS worker actually needs. Runnable standalone from a single examples/complete deployment, or composed piece by piece if you already have a VPC or cluster.

I built the node pool module first, after doing this same handful of steps by hand enough times across different clusters that it was clearly worth turning into a module. I added the VPC and cluster modules around it so the whole thing is a complete, terraform apply-able reference architecture rather than something that only works if you already have half the infrastructure in place.

What it creates

modules/vpc -- a VPC with public and private subnets across multiple AZs, an Internet Gateway, NAT Gateway(s) (single shared or one per AZ), and the kubernetes.io/cluster/<name> / kubernetes.io/role/elb tags EKS add-ons (cluster-autoscaler, the AWS Load Balancer Controller) rely on for subnet auto-discovery.

modules/eks-cluster -- the EKS control plane: cluster IAM role, the aws_eks_cluster resource itself, and an IAM OIDC provider for IRSA (IAM Roles for Service Accounts), so downstream add-ons get pod-scoped IAM instead of broad node-wide permissions.

modules/eks-gpu-nodepool -- the GPU-backed managed node group:

• An aws_launch_template with a configurable root volume size (model weights and container images are large), IMDSv2 enforced, and detailed monitoring on.
• An aws_eks_node_group (managed node group, not a hand-rolled ASG) using that launch template, with:
  • A nvidia.com/gpu=true:NoSchedule taint by default, so the scheduler won't place non-GPU pods here unless they explicitly tolerate it.
  • node-role/gpu and nvidia.com/gpu.present labels for node selectors.
  • Spot or on-demand capacity, your choice.
• Optionally, a dedicated IAM role with the four policies an EKS worker node needs (AmazonEKSWorkerNodePolicy, AmazonEKS_CNI_Policy, AmazonEC2ContainerRegistryReadOnly, AmazonSSMManagedInstanceCore) -- or pass in an existing role if your org already has one.

Why three separate modules instead of one

VPCs and EKS control planes change rarely; GPU node pools get added, resized, and retired far more often. Keeping them as separate, composable modules means adding a new GPU pool to an existing cluster doesn't require re-planning the network or control plane -- you use just modules/eks-gpu-nodepool on its own, which is what examples/basic demonstrates. examples/complete shows all three composed together for a from-scratch deployment. This mirrors how the well-known community modules split things up (e.g. terraform-aws-modules/eks handles the cluster; node groups are typically separate).

Design notes

• Managed node group, not a self-managed ASG. EKS handles node draining, AMI upgrades, and lifecycle hooks for managed node groups. Reimplementing that in Terraform/user-data is extra surface area for very little benefit at this scope.
• ignore_changes on desired_size. If you're running cluster-autoscaler (or Karpenter) alongside this, it will mutate the ASG's desired size directly. Without ignore_changes, every terraform plan would show a diff fighting the autoscaler. This module ignores that field on purpose -- Terraform owns the shape of the node group (instance types, taints, IAM), the autoscaler owns how many.
• Taint on by default, not opt-in. GPU instances are expensive enough that "a pod accidentally landed here" is a real cost, not just an inconvenience. apply_gpu_taint = false is there if you have a reason to disable it, but the default is the safer one.
• Spot is a first-class option, not bolted on. Inference/batch ML workloads on autoscaled pools are often fine with spot interruption (2 minutes notice, pods reschedule); the example shows spot for exactly that reason, with a comment on when you'd want on-demand instead.

Usage

Option A: deploy everything from scratch

cd examples/complete
terraform init
terraform apply -var="cluster_name=gpu-inference-demo"

This provisions a VPC, an EKS cluster, and a spot GPU node pool, and prints the aws eks update-kubeconfig command to connect kubectl. See examples/complete for all the variables you can override (instance types, capacity type, subnet CIDRs, single vs. per-AZ NAT, etc).

Option B: attach a GPU node pool to a cluster you already have

module "gpu_node_pool" {
  source = "github.com/milind2/terraform-aws-eks-gpu-nodepool//modules/eks-gpu-nodepool"

  cluster_name    = "my-eks-cluster"
  node_group_name = "inference-gpu-spot"
  subnet_ids      = ["subnet-0123456789abcdef0", "subnet-0fedcba9876543210"]

  instance_types = ["g5.xlarge", "g5.2xlarge"]
  capacity_type  = "SPOT"

  desired_size = 1
  min_size     = 0
  max_size     = 6
}

See examples/basic for the complete runnable version of this, including provider configuration.

Inputs (eks-gpu-nodepool module)

Name	Description	Type	Default
cluster_name	Existing EKS cluster to attach to	string	required
node_group_name	Name for the node group	string	required
subnet_ids	Subnets to launch nodes into	list(string)	required
instance_types	GPU instance types	list(string)	["g5.xlarge"]
ami_type	AL2_x86_64_GPU or BOTTLEROCKET_x86_64_NVIDIA	string	"AL2_x86_64_GPU"
capacity_type	ON_DEMAND or SPOT	string	"ON_DEMAND"
desired_size / min_size / max_size	Node group scaling config	number	1 / 0 / 4
disk_size_gb	Root EBS volume size	number	200
apply_gpu_taint	Taint nodes so only GPU workloads schedule here	bool	true
labels	Extra Kubernetes node labels	map(string)	{}
create_iam_role	Create a dedicated node IAM role	bool	true
node_role_arn	Existing role ARN (when create_iam_role = false)	string	null
tags	Tags applied to all resources	map(string)	{}

Outputs

Name	Description
node_group_arn	ARN of the created node group
node_group_status	Current node group status
launch_template_id	ID of the backing launch template
iam_role_arn	ARN of the node IAM role in use
node_labels	Full set of Kubernetes labels applied to nodes

Repo layout

modules/
  vpc/               VPC, public/private subnets, NAT, EKS discovery tags
  eks-cluster/       EKS control plane, cluster IAM role, OIDC provider (IRSA)
  eks-gpu-nodepool/  GPU-tainted managed node group (the original module)
examples/
  basic/             Attach a GPU node pool to a cluster you already have
  complete/          Full VPC + EKS cluster + GPU node pool, from scratch

What I'd add next

• Karpenter NodePool/EC2NodeClass variant, since Karpenter is increasingly the default over cluster-autoscaler + managed node groups for GPU-heavy, bursty workloads.
• GCP GKE cluster + node pool equivalents, to match a genuine multi-cloud story rather than just AWS.
• An optional nvidia-device-plugin DaemonSet manifest and core EKS add-ons (VPC CNI, CoreDNS, kube-proxy as managed add-ons) bundled into examples/complete, so the cluster is fully schedulable out of the box instead of needing a couple of kubectl applys first.
• A default (non-GPU) system node group in eks-cluster or examples/complete for CoreDNS/cluster add-ons, so they're not competing for space with tainted GPU nodes.

License

MIT