You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: chapters/appendices.md
+11-2Lines changed: 11 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -162,9 +162,18 @@ What interviewers listen for, by tier:
162
162
| 3 — Design/Debug | Failure-mode and scale instincts | Asks clarifying questions, bisects systematically, sizes things (objects × bytes, qps), states blast radius and recovery | Jumps to one tool, ignores scale numbers, no failure story |
163
163
| 4 — Judge (principal+) | Consequence, economics, strategy | Frames the org and fleet consequence, names assumptions and kill-criteria, argues the strongest counter-position before committing, cites history ("we tried X, it failed because…") | Mechanism-perfect but consequence-free; "it depends" with no decision; trend name-dropping without a position |
164
164
165
-
Five signals that run across all tiers: (1) mechanism accuracy; (2) naming the acting component; (3) trade-off awareness; (4) failure-mode instincts — unprompted "and if that's down…"; (5) scale instincts — unprompted "and at 10k objects…".
165
+
Five signals that run across all tiers:
166
166
167
-
Two more mark principal-level answers — and a staff candidate who shows them signals the next rung: (6) economic framing — cost, people, and risk enter the answer unprompted; (7) kill-criteria — the answer states what evidence would change the recommendation.
167
+
1. Mechanism accuracy.
168
+
2. Naming the acting component.
169
+
3. Trade-off awareness.
170
+
4. Failure-mode instincts — unprompted "and if that's down…".
171
+
5. Scale instincts — unprompted "and at 10k objects…".
172
+
173
+
Two more mark principal-level answers — and a staff candidate who shows them signals the next rung:
174
+
175
+
6. Economic framing — cost, people, and risk enter the answer unprompted.
176
+
7. Kill-criteria — the answer states what evidence would change the recommendation.
168
177
169
178
**Weak vs strong, same question** — "What happens when a liveness probe fails?"
Copy file name to clipboardExpand all lines: chapters/ch01.md
+20-2Lines changed: 20 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,19 @@ Interviewers open with architecture to check your mental model, because every de
6
6
7
7
## Concepts
8
8
9
-
**Control plane vs data plane.** The control plane decides what should run: kube-apiserver (the only component that talks to etcd), etcd (the strongly consistent key-value store holding all cluster state), kube-scheduler (assigns pods to nodes), kube-controller-manager or KCM (one binary bundling dozens of controllers: ReplicaSet, node lifecycle, EndpointSlice, garbage collection), and cloud-controller-manager (load balancers, node instances). The data plane runs it, on every node: kubelet (the node agent that makes pods real), a container runtime such as containerd (driven over CRI, a gRPC contract), and kube-proxy (programs Service routing rules).
9
+
**Control plane vs data plane.** The control plane decides what should run:
10
+
11
+
-**kube-apiserver:** the only component that talks to etcd.
12
+
-**etcd:** the strongly consistent key-value store holding all cluster state.
13
+
-**kube-scheduler:** assigns pods to nodes.
14
+
-**kube-controller-manager or KCM:** one binary bundling dozens of controllers: ReplicaSet, node lifecycle, EndpointSlice, garbage collection.
-**kubelet:** the node agent that makes pods real.
20
+
-**container runtime:** such as containerd, driven over CRI, a gRPC contract.
21
+
-**kube-proxy:** programs Service routing rules.
10
22
11
23
**Hub-and-spoke.** Components never call each other directly. The scheduler does not call the kubelet; it writes a binding, and the kubelet notices via a watch (a streaming subscription to object changes). This makes the API server the scaling and security choke point, and it is why a dead control plane leaves running workloads untouched (Chapter 10).
12
24
@@ -125,7 +137,13 @@ sequenceDiagram
125
137
126
138
**Q 1.4 — Why do components communicate only through the API server instead of calling each other?**
127
139
128
-
**Answer.** Hub-and-spoke buys decoupling, security, and recoverability. Decoupling: components only understand objects, so each can be replaced independently (custom schedulers, virtual kubelets). Security: one place enforces authn, RBAC, admission, and audit for every state change. Recoverability: state lives in etcd, not in transit — a restarting component relists and catches up; a partitioned one reconciles late. The cost: the API server becomes the throughput bottleneck, which is why the watch cache and API Priority and Fairness exist (Chapter 2).
140
+
**Answer.** Hub-and-spoke buys decoupling, security, and recoverability.
141
+
142
+
-**Decoupling:** components only understand objects, so each can be replaced independently (custom schedulers, virtual kubelets).
143
+
-**Security:** one place enforces authn, RBAC, admission, and audit for every state change.
144
+
-**Recoverability:** state lives in etcd, not in transit — a restarting component relists and catches up; a partitioned one reconciles late.
145
+
146
+
The cost: the API server becomes the throughput bottleneck, which is why the watch cache and API Priority and Fairness exist (Chapter 2).
129
147
130
148
*Strong answers also mention:* this design makes "control plane down" non-fatal for running workloads.
Copy file name to clipboardExpand all lines: chapters/ch02.md
+18-3Lines changed: 18 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -23,19 +23,34 @@ flowchart TD
23
23
```
24
24
*Figure 2.1 — the admission chain sits between authorization and storage; a veto anywhere means nothing is persisted.*
25
25
26
-
**Admission: webhooks and CEL policies.** Webhooks are HTTPS callbacks to services you run; the API server blocks the request on them, governed by `timeoutSeconds` (max 30s) and `failurePolicy` (Flow 4). ValidatingAdmissionPolicy (GA since v1.30) instead evaluates CEL expressions *inside* the API server: no network hop, no availability risk. MutatingAdmissionPolicy followed (GA in v1.36). Rule of thumb: use CEL unless you need external data or complex mutation.
26
+
**Admission: webhooks and CEL policies.** Three mechanisms enforce admission rules:
27
+
28
+
-**Webhooks:** HTTPS callbacks to services you run; the API server blocks the request on them, governed by `timeoutSeconds` (max 30s) and `failurePolicy` (Flow 4).
29
+
-**ValidatingAdmissionPolicy:** evaluates CEL expressions *inside* the API server (GA since v1.30); no network hop, no availability risk.
30
+
-**MutatingAdmissionPolicy:** similar to validating, for mutations (GA in v1.36).
31
+
32
+
Rule of thumb: use CEL unless you need external data or complex mutation.
27
33
28
34
**etcd essentials.** etcd is a raft-replicated key-value store: one elected leader orders all writes; a quorum (majority) must acknowledge each. Every write increments a global, monotonic **revision**, and etcd keeps multi-version history, so it can answer "what changed since revision X" — the watch primitive. History is finite: **compaction** (triggered periodically by the API server) discards old revisions; a watcher asking for a compacted revision gets the error clients see as "too old resource version" (Flow 3).
29
35
30
-
**resourceVersion and optimistic concurrency.** An object's resourceVersion is the etcd revision of its last modification — an opaque string, never to be parsed or compared arithmetically. Updates carry the resourceVersion the client last read; if the object changed meanwhile, the API server returns 409 Conflict and the client must re-read and retry (Flow 2). On list/watch, resourceVersion selects consistency: unset means a fresh quorum-consistent read, `0` means any cached state. An exact value means "start after this point" for a watch, but "at least this fresh" for a list — the server may return newer data.
36
+
**resourceVersion and optimistic concurrency.** An object's resourceVersion is the etcd revision of its last modification — an opaque string, never to be parsed or compared arithmetically. Updates carry the resourceVersion the client last read; if the object changed meanwhile, the API server returns 409 Conflict and the client must re-read and retry (Flow 2). On list/watch, resourceVersion selects consistency:
37
+
38
+
-**unset:** a fresh quorum-consistent read.
39
+
-**`0`:** any cached state.
40
+
-**exact value:** "start after this point" for a watch, but "at least this fresh" for a list — the server may return newer data.
31
41
32
42
**Server-Side Apply (SSA).** GA since v1.22. The *server* merges changes and tracks, per field, which **field manager** (a named client) owns it — recorded in `managedFields`. Applying means: "make the fields I mention match, and I own them." A field you stop mentioning is removed from the object if you were its sole owner; if it is co-owned, only your claim is released and the value stays. Two managers claiming the same field produce an explicit conflict, resolved deliberately (`force=true` takes ownership). This replaces fragile client-side merges and makes multi-controller ownership of one object safe.
33
43
34
44
**The watch cache.** The API server keeps one etcd watch per resource type and an in-memory cache of recent versions. Nearly all client watches and many lists are served from this cache — this is what lets thousands of kubelets and controllers watch pods without melting etcd. The cache holds a bounded history window; clients that fall behind must relist (Flow 3). Periodic **bookmark** events keep idle watchers' resume points fresh.
35
45
36
46
**API Priority and Fairness (APF).** GA since v1.29. Requests are classified by FlowSchemas into priority levels, each with concurrency shares ("seats") and fair queuing across distinct flows — so a misbehaving controller cannot starve kubelet heartbeats. Throttled requests get 429; the `apiserver_flowcontrol_*` metrics show what is queued or rejected.
37
47
38
-
**CRDs vs aggregation.** CRDs declare new resource types the API server itself serves and stores in etcd — no code to run; behavior comes from your controller. An aggregated API (APIService) proxies a URL prefix to your own server, which may use custom storage and semantics (metrics server is the classic example). Default to CRDs; aggregate only when you need semantics CRDs cannot express.
48
+
**CRDs vs aggregation.**
49
+
50
+
-**CRDs:** declare new resource types the API server itself serves and stores in etcd — no code to run; behavior comes from your controller.
51
+
-**Aggregated API (APIService):** proxies a URL prefix to your own server, which may use custom storage and semantics (metrics server is the classic example).
52
+
53
+
Default to CRDs; aggregate only when you need semantics CRDs cannot express.
39
54
40
55
**ServiceAccounts, tokens, and RBAC mechanics.** Every pod authenticates as a ServiceAccount. Its token is *projected*: the kubelet requests it from the TokenRequest API — audience-scoped, bound to the pod, expiring (default 1h), refreshed before expiry. No stored Secret is involved; auto-created Secret tokens ended in v1.24. On each request the API server validates signature, expiry, audience, and that the bound pod still exists, so a leaked token dies with its pod. On the authorization side, RBAC is deny-by-default and purely additive: ClusterRoles can *aggregate* (a controller merges labeled roles into one), and escalation is prevented — you cannot grant permissions you do not hold, gated by the `escalate` and `bind` verbs.
Copy file name to clipboardExpand all lines: chapters/ch03.md
+39-4Lines changed: 39 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,13 +6,32 @@ Scheduling questions test whether you know what the scheduler actually does —
6
6
7
7
## Concepts
8
8
9
-
**The scheduling framework.** The scheduler is a plugin pipeline. A **scheduling cycle** runs one pod at a time through: PreFilter (precompute), Filter (which nodes are feasible), PostFilter (runs only when none are — preemption lives here), Score (rank feasible nodes), Reserve (claim resources in the scheduler's cache), Permit (approve, reject, or wait). Then an asynchronous **binding cycle** — PreBind, Bind, PostBind — runs in a goroutine so the next pod's scheduling cycle starts immediately. Default behaviors (resource fit, taints, affinity, spreading) are themselves plugins.
9
+
**The scheduling framework.** The scheduler is a plugin pipeline. A **scheduling cycle** runs one pod at a time through:
10
10
11
-
**Queues.** Pods awaiting scheduling live in three structures: **activeQ** (priority-ordered, ready to schedule), **backoffQ** (failed recently; waiting out exponential backoff), and the **unschedulable set** (no fit; parked). Cluster events that could change the answer — node added, pod deleted, taint removed — move parked pods back toward activeQ; the scheduler tracks which event types can help which pod, so it doesn't retry blindly.
11
+
1. PreFilter (precompute)
12
+
2. Filter (which nodes are feasible)
13
+
3. PostFilter (runs only when none are — preemption lives here)
14
+
4. Score (rank feasible nodes)
15
+
5. Reserve (claim resources in the scheduler's cache)
16
+
6. Permit (approve, reject, or wait)
17
+
18
+
An asynchronous **binding cycle** — PreBind, Bind, PostBind — follows in a goroutine so the next pod's scheduling cycle starts immediately. Default behaviors (resource fit, taints, affinity, spreading) are themselves plugins.
19
+
20
+
**Queues.** Pods awaiting scheduling live in three structures:
21
+
22
+
-**activeQ:** priority-ordered, ready to schedule.
23
+
-**backoffQ:** failed recently; waiting out exponential backoff.
24
+
-**unschedulable set:** no fit; parked.
25
+
26
+
Cluster events that could change the answer — node added, pod deleted, taint removed — move parked pods back toward activeQ; the scheduler tracks which event types can help which pod, so it doesn't retry blindly.
12
27
13
28
**Requests, not usage.** Filtering and scoring use pod *requests* against node *allocatable* — never live metrics. On large clusters the scheduler stops searching for feasible nodes once it finds enough (`percentageOfNodesToScore`), then scores only those, to bound latency.
14
29
15
-
**Placement constraints.** Node affinity selects nodes by labels (required rules filter, preferred rules score). Pod affinity/anti-affinity constrains placement relative to other pods — powerful but expensive, since it examines existing pods per topology domain. Topology spread constraints (`maxSkew` across a topology key) are the scalable way to spread replicas across zones or nodes.
-**Pod affinity/anti-affinity:** constrains placement relative to other pods — powerful but expensive, since it examines existing pods per topology domain.
34
+
-**Topology spread constraints** (`maxSkew` across a topology key): the scalable way to spread replicas across zones or nodes.
16
35
17
36
```mermaid
18
37
flowchart TD
@@ -181,7 +200,23 @@ sequenceDiagram
181
200
182
201
**Q 3.1 — Walk me through the scheduling framework's extension points for one pod.**
183
202
184
-
**Answer.** Scheduling cycle, serial per pod: PreEnqueue and QueueSort control queue entry and order; PreFilter precomputes; Filter prunes infeasible nodes; PostFilter runs only on failure (preemption); Score ranks; Reserve claims cache resources; Permit can approve, deny, or hold. Binding cycle, asynchronous: PreBind (volume readiness), Bind, PostBind. Every default behavior is a plugin on these same hooks — extending the scheduler means writing plugins, not forking it.
203
+
**Answer.** Scheduling cycle, serial per pod:
204
+
205
+
1. PreEnqueue and QueueSort control queue entry and order.
206
+
2. PreFilter precomputes.
207
+
3. Filter prunes infeasible nodes.
208
+
4. PostFilter runs only on failure (preemption).
209
+
5. Score ranks.
210
+
6. Reserve claims cache resources.
211
+
7. Permit can approve, deny, or hold.
212
+
213
+
Binding cycle, asynchronous:
214
+
215
+
1. PreBind (volume readiness).
216
+
2. Bind.
217
+
3. PostBind.
218
+
219
+
Every default behavior is a plugin on these same hooks — extending the scheduler means writing plugins, not forking it.
185
220
186
221
*Strong answers also mention:* Unreserve as the rollback path, and why binding is async — scheduling throughput continues while binds do I/O.
0 commit comments