@@ -79,15 +79,20 @@
    "source": [
     "## What is a Machine Learning Model?\n",
     "\n",
-    "At a high level, a **model** is a rule that maps inputs to outputs:\n",
+    "At a high level, a **model** is a rule that maps inputs to outputs.\n",
     "\n",
+    "::: {.callout-note title=\"Definition: a model\"}\n",
     "$$\n",
     "\\hat{y} = f(\\mathbf{x})\n",
     "$$\n",
     "\n",
-    "- $\\mathbf{x}$: input (features)\n",
-    "- $\\hat{y}$: model output (prediction)\n",
-    "- $f$: model (the rule we learn from data)\n",
+    "- $\\mathbf{x}$: input (features), one example as a point in $\\mathbb{R}^d$,\n",
+    "- $\\hat{y}$: model output (prediction),\n",
+    "- $f$: the model, i.e. the rule we learn from data.\n",
+    "\n",
+    "A model is a *function*. Building one means choosing its form, then choosing the\n",
+    "numbers inside it.\n",
+    ":::\n",
     "\n",
     "In practice, we will write:\n",
     "\n",
@@ -103,9 +108,13 @@
     "- $\\boldsymbol{\\eta}$: **hyperparameters** (we choose these *before* training)\n",
     "  - e.g. number of neighbors `k`, depth of a tree, regularization strength\n",
     "\n",
-    "::: {.callout-note title=\"Key idea\"}\n",
-    "- Parameters are *learned* from data by the training algorithm.\n",
-    "- Hyperparameters are *chosen* by us to control model complexity and behavior.\n",
+    "::: {.callout-tip title=\"Key concept: parameters are learned, hyperparameters are chosen\"}\n",
+    "- **Parameters** $\\mathbf{w}$ are *learned* from data by the training algorithm.\n",
+    "- **Hyperparameters** $\\boldsymbol{\\eta}$ are *chosen* by us, before training, and\n",
+    "  control the model's complexity and behavior.\n",
+    "\n",
+    "The notation $\\mathbf{w}(\\boldsymbol{\\eta})$ says this out loud: which parameters we\n",
+    "end up learning depends on the hyperparameters we picked first.\n",
     ":::\n",
     "\n",
     "Later we will tune hyperparameters more carefully. For now, we’ll use simple defaults and explore how they affect behavior.\n"
@@ -142,6 +151,13 @@
     "\n",
     "> “Let the data speak; find structure in $\\mathbf{x}$ itself.”\n",
     "\n",
+    "::: {.callout-tip title=\"Key concept: the task comes from the question, not the data\"}\n",
+    "The same dataset matrix $X$ can serve any of these tasks. What changes is what we ask\n",
+    "for: a number (regression), a label (classification), or structure in $X$ itself\n",
+    "(unsupervised). That single choice fixes the target, the family of models we can use,\n",
+    "*and* the metric we are allowed to judge the result by.\n",
+    ":::\n",
+    "\n",
     "In this module, we focus on **supervised learning and classification** to get familiar with modeling in `scikit-learn`.\n"
    ]
   },
@@ -306,7 +322,19 @@
     "2. _fit_: train data X, labels y (labels needed for supervised models only)\n",
     "    - `model.fit(X, y)`\n",
     "3. _predict_: generates output\n",
-    "    - `model.predict(X_test)`"
+    "    - `model.predict(X_test)`\n",
+    "\n",
+    "::: {.callout-note title=\"Key concept: every sklearn model is the same three calls\"}\n",
+    "```python\n",
+    "model = Estimator(hyperparameters)   # 1. pick the model family and its knobs\n",
+    "model.fit(X, y)                      # 2. learn the parameters from data\n",
+    "y_pred = model.predict(X_new)        # 3. predict on new inputs\n",
+    "```\n",
+    "\n",
+    "The estimator changes; the three lines do not. Everything passed to the constructor is\n",
+    "a hyperparameter $\\boldsymbol{\\eta}$, and `fit` is where the parameters $\\mathbf{w}$\n",
+    "come from.\n",
+    ":::"
    ]
   },
   {
@@ -334,7 +362,19 @@
     "2. Find the $k$ closest training points.\n",
     "3. Let those neighbors vote.\n",
     "\n",
-    "There is almost no training phase. The model mostly stores the training data and uses distance at prediction time.\n"
+    "There is almost no training phase. The model mostly stores the training data and uses distance at prediction time.\n",
+    "\n",
+    "::: {.callout-note title=\"Definition: the $k$-nearest-neighbors rule\"}\n",
+    "To predict the label of a new point $\\mathbf{x}$:\n",
+    "\n",
+    "1. compute the distance $d(\\mathbf{x}, \\mathbf{x}_i)$ to every training point,\n",
+    "2. keep the $k$ smallest, giving the neighbor set $N_k(\\mathbf{x})$,\n",
+    "3. predict the majority label among those $k$ neighbors.\n",
+    "\n",
+    "There is no fitted $\\mathbf{w}$ here: kNN stores the training set and does all its work\n",
+    "at prediction time. Its behavior is set entirely by hyperparameters, $k$ and the\n",
+    "distance $d$ (the $L^p$ distance from Notebook 1a).\n",
+    ":::\n"
    ]
   },
   {
@@ -2038,6 +2078,18 @@
     "- `p=1` gives Manhattan distance,\n",
     "- `p=2` gives Euclidean distance.\n",
     "\n",
+    "::: {.callout-important title=\"Key concept: $k$ controls model complexity\"}\n",
+    "$k$ sets how much the model is allowed to trust a single point.\n",
+    "\n",
+    "- $k = 1$: every training point rules its own neighborhood. Training accuracy is\n",
+    "  perfect and the boundary chases noise, i.e. **overfitting**.\n",
+    "- $k$ close to $n$: every prediction is the majority label of nearly the whole dataset.\n",
+    "  The boundary is so smooth it ignores real structure, i.e. **underfitting**.\n",
+    "\n",
+    "Nothing in the data tells us the right $k$: we have to *choose* it. Notebook 2b is\n",
+    "about how to make that choice honestly.\n",
+    ":::\n",
+    "\n",
     "Run the next cell, then move the sliders.\n"
    ]
   },
@@ -2192,6 +2244,40 @@
     "4. Compare its boundary to kNN.\n",
     "5. Which model seems more flexible? Which seems easier to explain?"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8ea28411",
+   "metadata": {
+    "id": "nb2a-takeaways"
+   },
+   "source": [
+    "## Takeaways\n",
+    "\n",
+    "::: {.callout-tip title=\"Takeaways from this lecture\"}\n",
+    "1. **A model is a function**, $\\hat{y} = f(\\mathbf{x}, \\mathbf{w}(\\boldsymbol{\\eta}))$,\n",
+    "   from features to a prediction, not a black box.\n",
+    "2. **Parameters vs. hyperparameters:** $\\mathbf{w}$ is *learned* by fitting;\n",
+    "   $\\boldsymbol{\\eta}$ is *chosen* by us before training.\n",
+    "3. **The task comes from the question.** The same $X$ supports regression (predict a\n",
+    "   number), classification (predict a label), and unsupervised learning (find structure\n",
+    "   in $X$ alone).\n",
+    "4. **Every sklearn model is the same three calls:** construct with hyperparameters,\n",
+    "   `fit` to learn parameters, `predict` on new inputs.\n",
+    "5. **kNN predicts by neighborhood vote:** distances to all training points, keep the $k$\n",
+    "   closest, take the majority label. It stores data instead of fitting parameters.\n",
+    "6. **The distance is a hyperparameter too:** Minkowski `p=1` is Manhattan and `p=2` is\n",
+    "   Euclidean, the $L^p$ distances from Notebook 1a, and they give different boundaries\n",
+    "   on the same data.\n",
+    "7. **$k$ controls complexity:** small $k$ gives a jagged boundary that overfits, large\n",
+    "   $k$ gives a smooth one that underfits. Notebook 2b turns this into a principled\n",
+    "   choice.\n",
+    "8. **A decision boundary makes a model's assumptions visible.** Plot it whenever you\n",
+    "   want to see what a classifier actually believes.\n",
+    "9. **A model encodes assumptions about which patterns matter.** kNN assumes nearby\n",
+    "   points share labels; that is a claim about the data, not a fact.\n",
+    ":::"
+   ]
   }
  ],
  "metadata": {
 
@@ -54,15 +54,20 @@
 # %% [markdown] id="39usOoiAe5DI"
 # ## What is a Machine Learning Model?
 #
-# At a high level, a **model** is a rule that maps inputs to outputs:
+# At a high level, a **model** is a rule that maps inputs to outputs.
 #
+# ::: {.callout-note title="Definition: a model"}
 # $$
 # \hat{y} = f(\mathbf{x})
 # $$
 #
-# - $\mathbf{x}$: input (features)
-# - $\hat{y}$: model output (prediction)
-# - $f$: model (the rule we learn from data)
+# - $\mathbf{x}$: input (features), one example as a point in $\mathbb{R}^d$,
+# - $\hat{y}$: model output (prediction),
+# - $f$: the model, i.e. the rule we learn from data.
+#
+# A model is a *function*. Building one means choosing its form, then choosing the
+# numbers inside it.
+# :::
 #
 # In practice, we will write:
 #
@@ -78,9 +83,13 @@
 # - $\boldsymbol{\eta}$: **hyperparameters** (we choose these *before* training)
 #   - e.g. number of neighbors `k`, depth of a tree, regularization strength
 #
-# ::: {.callout-note title="Key idea"}
-# - Parameters are *learned* from data by the training algorithm.
-# - Hyperparameters are *chosen* by us to control model complexity and behavior.
+# ::: {.callout-tip title="Key concept: parameters are learned, hyperparameters are chosen"}
+# - **Parameters** $\mathbf{w}$ are *learned* from data by the training algorithm.
+# - **Hyperparameters** $\boldsymbol{\eta}$ are *chosen* by us, before training, and
+#   control the model's complexity and behavior.
+#
+# The notation $\mathbf{w}(\boldsymbol{\eta})$ says this out loud: which parameters we
+# end up learning depends on the hyperparameters we picked first.
 # :::
 #
 # Later we will tune hyperparameters more carefully. For now, we’ll use simple defaults and explore how they affect behavior.
@@ -111,6 +120,13 @@
 #
 # > “Let the data speak; find structure in $\mathbf{x}$ itself.”
 #
+# ::: {.callout-tip title="Key concept: the task comes from the question, not the data"}
+# The same dataset matrix $X$ can serve any of these tasks. What changes is what we ask
+# for: a number (regression), a label (classification), or structure in $X$ itself
+# (unsupervised). That single choice fixes the target, the family of models we can use,
+# *and* the metric we are allowed to judge the result by.
+# :::
+#
 # In this module, we focus on **supervised learning and classification** to get familiar with modeling in `scikit-learn`.
 #
 
@@ -213,6 +229,18 @@
 #     - `model.fit(X, y)`
 # 3. _predict_: generates output
 #     - `model.predict(X_test)`
+#
+# ::: {.callout-note title="Key concept: every sklearn model is the same three calls"}
+# ```python
+# model = Estimator(hyperparameters)   # 1. pick the model family and its knobs
+# model.fit(X, y)                      # 2. learn the parameters from data
+# y_pred = model.predict(X_new)        # 3. predict on new inputs
+# ```
+#
+# The estimator changes; the three lines do not. Everything passed to the constructor is
+# a hyperparameter $\boldsymbol{\eta}$, and `fit` is where the parameters $\mathbf{w}$
+# come from.
+# :::
 
 # %% [markdown] id="sUGXrwTJu0DG"
 # A list of sklearn supervised learning methods can be found [here](https://scikit-learn.org/stable/supervised_learning.html).
@@ -231,6 +259,18 @@
 #
 # There is almost no training phase. The model mostly stores the training data and uses distance at prediction time.
 #
+# ::: {.callout-note title="Definition: the $k$-nearest-neighbors rule"}
+# To predict the label of a new point $\mathbf{x}$:
+#
+# 1. compute the distance $d(\mathbf{x}, \mathbf{x}_i)$ to every training point,
+# 2. keep the $k$ smallest, giving the neighbor set $N_k(\mathbf{x})$,
+# 3. predict the majority label among those $k$ neighbors.
+#
+# There is no fitted $\mathbf{w}$ here: kNN stores the training set and does all its work
+# at prediction time. Its behavior is set entirely by hyperparameters, $k$ and the
+# distance $d$ (the $L^p$ distance from Notebook 1a).
+# :::
+#
 
 # %%
 # The kNN computation: distances to every training point, the k closest, and
@@ -454,6 +494,18 @@ def plot_decision_boundary(clf, X, y):
 # - `p=1` gives Manhattan distance,
 # - `p=2` gives Euclidean distance.
 #
+# ::: {.callout-important title="Key concept: $k$ controls model complexity"}
+# $k$ sets how much the model is allowed to trust a single point.
+#
+# - $k = 1$: every training point rules its own neighborhood. Training accuracy is
+#   perfect and the boundary chases noise, i.e. **overfitting**.
+# - $k$ close to $n$: every prediction is the majority label of nearly the whole dataset.
+#   The boundary is so smooth it ignores real structure, i.e. **underfitting**.
+#
+# Nothing in the data tells us the right $k$: we have to *choose* it. Notebook 2b is
+# about how to make that choice honestly.
+# :::
+#
 # Run the next cell, then move the sliders.
 #
 
@@ -568,3 +620,30 @@ def plot_knn_model(n_neighbors=5, p=2, weights="uniform"):
 # 3. Report train accuracy and test accuracy.
 # 4. Compare its boundary to kNN.
 # 5. Which model seems more flexible? Which seems easier to explain?
+
+# %% [markdown] id="nb2a-takeaways"
+# ## Takeaways
+#
+# ::: {.callout-tip title="Takeaways from this lecture"}
+# 1. **A model is a function**, $\hat{y} = f(\mathbf{x}, \mathbf{w}(\boldsymbol{\eta}))$,
+#    from features to a prediction, not a black box.
+# 2. **Parameters vs. hyperparameters:** $\mathbf{w}$ is *learned* by fitting;
+#    $\boldsymbol{\eta}$ is *chosen* by us before training.
+# 3. **The task comes from the question.** The same $X$ supports regression (predict a
+#    number), classification (predict a label), and unsupervised learning (find structure
+#    in $X$ alone).
+# 4. **Every sklearn model is the same three calls:** construct with hyperparameters,
+#    `fit` to learn parameters, `predict` on new inputs.
+# 5. **kNN predicts by neighborhood vote:** distances to all training points, keep the $k$
+#    closest, take the majority label. It stores data instead of fitting parameters.
+# 6. **The distance is a hyperparameter too:** Minkowski `p=1` is Manhattan and `p=2` is
+#    Euclidean, the $L^p$ distances from Notebook 1a, and they give different boundaries
+#    on the same data.
+# 7. **$k$ controls complexity:** small $k$ gives a jagged boundary that overfits, large
+#    $k$ gives a smooth one that underfits. Notebook 2b turns this into a principled
+#    choice.
+# 8. **A decision boundary makes a model's assumptions visible.** Plot it whenever you
+#    want to see what a classifier actually believes.
+# 9. **A model encodes assumptions about which patterns matter.** kNN assumes nearby
+#    points share labels; that is a claim about the data, not a fact.
+# :::