MCO-2599: makes bootc-operator aware of karpenter (backport #204)

AGENTS.md

@@ -66,6 +66,11 @@ commit and especially when the agent thinks a task is complete
 to spawn a subagent to perform a review using guidelines (alongside
 looking for any other issues).
 
+### Testing
+
+Run this repository's unit tests with `make unit`. This is required to ensure
+that envtest is set up and running for tests that need it.
+
 If the agent is performing a review of other's code, the same
 principles apply.
 

api/v1alpha1/constants.go

@@ -30,4 +30,51 @@ const (
 	// state for which the controller considered emitting an event. It is
 	// observability bookkeeping only and is never used to drive reconciliation.
 	AnnotationLastObservedState = "bootc.dev/last-observed-state"
+
+	// AnnotationOwnsKarpenterDoNotRepair is set on a BootcNode to record that
+	// the controller added the karpenter.sh/do-not-repair annotation to the K8s
+	// Node during slot assignment. When present, the controller removes the
+	// karpenter.sh/do-not-repair annotation from the Node when the slot is freed.
+	// It is not set when the annotation was already present before slot assignment
+	// (externally set), so the controller never removes an annotation it did not add.
+	AnnotationOwnsKarpenterDoNotRepair = "bootc.dev/owns-karpenter-do-not-repair"
+
+	// AnnotationOwnsKarpenterDoNotDisrupt is set on a BootcNode to record that
+	// the controller added karpenter.sh/do-not-disrupt to the K8s Node during
+	// slot assignment. It has the same ownership and cleanup semantics as
+	// AnnotationOwnsKarpenterDoNotRepair.
+	AnnotationOwnsKarpenterDoNotDisrupt = "bootc.dev/owns-karpenter-do-not-disrupt"
+
+	// AnnotationOwnsKarpenterNodeClaimDoNotRepair records that the controller
+	// added do-not-repair to the associated Karpenter NodeClaim. It follows the
+	// same ownership and cleanup semantics as AnnotationOwnsKarpenterDoNotRepair.
+	AnnotationOwnsKarpenterNodeClaimDoNotRepair = "bootc.dev/owns-karpenter-nodeclaim-do-not-repair"
+
+	// AnnotationOwnsKarpenterNodeClaimDoNotDisrupt records that the controller
+	// added do-not-disrupt to the associated Karpenter NodeClaim. It follows the
+	// same ownership and cleanup semantics as AnnotationOwnsKarpenterDoNotRepair.
+	AnnotationOwnsKarpenterNodeClaimDoNotDisrupt = "bootc.dev/owns-karpenter-nodeclaim-do-not-disrupt"
+
+	// TaintKeyOsNotReady is a well-known taint key that cluster administrators
+	// can apply to Karpenter NodePools as a startupTaint to gate workload
+	// scheduling until bootc-operator has applied and verified the pool's target
+	// OS image. The full taint is "bootc.dev/os-not-ready:NoSchedule".
+	//
+	// When a node joins with this taint, bootc-operator records ownership via
+	// AnnotationOwnsOsNotReadyTaint and removes the taint once the node is
+	// UpToDate and Ready. If the node is already running the target image when
+	// it joins, the taint is removed without requiring a reboot slot.
+	//
+	// This taint MUST be declared as a startupTaint in the Karpenter NodePool.
+	// If it is configured only as a regular taint, Karpenter will treat nodes
+	// with the taint as unable to satisfy pending pods and will provision
+	// additional nodes in a loop.
+	TaintKeyOsNotReady = "bootc.dev/os-not-ready"
+
+	// AnnotationOwnsOsNotReadyTaint is set on a BootcNode to record that the
+	// controller found and owns the bootc.dev/os-not-ready:NoSchedule taint on
+	// the K8s Node at join time. When present, the controller removes the taint
+	// once the node is UpToDate and Ready. The controller never removes a taint
+	// it did not record ownership of.
+	AnnotationOwnsOsNotReadyTaint = "bootc.dev/owns-os-not-ready-taint"
 )

config/rbac/role.yaml

@@ -49,6 +49,15 @@ rules:
   - daemonsets
   verbs:
   - get
+- apiGroups:
+  - karpenter.sh
+  resources:
+  - nodeclaims
+  verbs:
+  - get
+  - list
+  - patch
+  - watch
 - apiGroups:
   - node.bootc.dev
   resources:

docs/design/ARCHITECTURE.md

@@ -427,6 +427,13 @@ governed by three rules:
    is bad. Note that Degraded is a BootcNode condition while Ready is
    a K8s Node condition -- these are independent checks.
 
+When assigning slots, the controller processes nodes that already have a slot
+first, then evaluates all other staged nodes until the available capacity is
+filled. A staged node that is temporarily deferred by an external condition
+does not consume capacity, so a later eligible node can still be assigned a
+slot. Existing slotted nodes continue to be processed even when no new slot is
+available.
+
 A node enters a slot when the controller sets the
 `bootc.dev/in-reboot-slot` annotation on the BootcNode, cordons the
 K8s Node, and starts draining. The annotation is the persistent marker

docs/src/SUMMARY.md

@@ -8,3 +8,5 @@
 - [Helm Chart](helm.md)
 - [Operations](operations/index.md)
   - [Managing a pool](operations/pool.md)
+  - [OS initialization taint](operations/os-not-ready-taint.md)
+  - [Karpenter integration](operations/karpenter.md)