You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 75e251b
Browse filesBrowse the repository at this point in the historyBrowse files
If a cluster contains both bootc-operator and Karpenter, it is possible
that Karpenter could reap cluster nodes while a bootc-operator driven
update is in progress. To prevent that, we detect whether the Node
object is owned by a Karpenter NodeClaim object. And in that situation,
we apply both the Karpenter Do Not Repair and Do Not Disrupt annotations
to each node when it is cordoned for an update and we remove it after
the update is completed.
Signed-off-by: Zack Zlotnik <zzlotnik@redhat.com>
Assisted-by: AI
2. The node proceeds through the normal update flow: drain, reboot, image
45
+
verification.
46
+
3. When the slot is freed (node is Ready and running the new image), the
47
+
controller checks the tracking annotations on the `BootcNode`. For each
48
+
annotation it owns, it removes the corresponding annotation from the Node
49
+
and clears the tracking annotation from the `BootcNode`. Both removals are
50
+
issued as a single Node patch.
51
+
52
+
## Pre-existing annotations
53
+
54
+
If either Karpenter annotation is already on the Node before slot assignment —
55
+
set by an external actor, another controller, or a manual operation —
56
+
bootc-operator leaves it completely alone:
57
+
58
+
- It does **not** overwrite or change the annotation on assignment.
59
+
- It does **not** set the corresponding tracking annotation on the `BootcNode`.
60
+
- It does **not** remove the annotation when the slot is freed.
61
+
62
+
The two annotations are tracked independently. If only one is pre-existing, the
63
+
operator owns (and later removes) only the other.
64
+
65
+
This ensures bootc-operator never removes an annotation it did not add.
66
+
67
+
## Limitations
68
+
69
+
`karpenter.sh/do-not-disrupt` only blocks voluntary disruption methods
70
+
(consolidation and drift). It does not protect a node against forceful
71
+
termination triggered by expiration (`expireAfter`), interruption (spot
72
+
instance reclaim), Node Auto Repair, or manual deletion. If any of those
73
+
events occur while a node is in a bootc-operator reboot slot, Karpenter may
74
+
still terminate it.
75
+
76
+
`karpenter.sh/do-not-repair` similarly has no effect on voluntary disruption
77
+
or expiration — it only suppresses Node Auto Repair.
78
+
79
+
If a node is terminated by expiration, interruption, or any other forceful
80
+
method while a bootc-operator update is in progress, the termination is caused
81
+
by Karpenter acting on its own policy — not by the bootc-operator update. The
82
+
update simply happened to be in flight at the time.
83
+
84
+
Refer to the [Karpenter disruption documentation](https://karpenter.sh/docs/concepts/disruption/)
85
+
for a full breakdown of which controls apply to which disruption methods.
86
+
87
+
## Non-Karpenter nodes
88
+
89
+
Nodes without a `NodeClaim` owner reference are unaffected. The feature
90
+
activates only when the owner reference check matches.
91
+
92
+
## Annotation reference
93
+
94
+
| Annotation | Object | Managed by | Purpose |
95
+
|---|---|---|---|
96
+
|`karpenter.sh/do-not-repair`| K8s Node | bootc-operator (or external) | Prevents Karpenter Node Auto Repair from replacing this node during an update |
97
+
|`karpenter.sh/do-not-disrupt`| K8s Node | bootc-operator (or external) | Prevents Karpenter voluntary disruption (consolidation, drift) during an update |
98
+
|`bootc.dev/owns-karpenter-do-not-repair`| BootcNode | bootc-operator | Tracks that the operator added `do-not-repair`; cleared when the annotation is removed on slot free |
99
+
|`bootc.dev/owns-karpenter-do-not-disrupt`| BootcNode | bootc-operator | Tracks that the operator added `do-not-disrupt`; cleared when the annotation is removed on slot free |
0 commit comments