|
1 | 1 | --- |
2 | 2 | title: Distillation |
3 | | -description: Compress a slow teacher into a tiny native student that runs anywhere. |
| 3 | +description: Compress a slow teacher into a tiny native student, then serve it like any other model. |
4 | 4 | --- |
5 | 5 |
|
6 | 6 | Foundation models are accurate but too heavy to call per query on CPU. The |
7 | | -answer is distillation: run the teacher once to label your data, then fit a small |
8 | | -**native student** that runs in the zero-dependency core with no runtime. |
| 7 | +answer is distillation: run the teacher once to label your data, then fit a |
| 8 | +small **native student** that runs in the zero-dependency core with no |
| 9 | +runtime. Every distilled student registers under the `student_id` you give |
| 10 | +it, and you serve it by passing that id as the `model` option of the normal |
| 11 | +serving call. There is no separate "serve a student" API. |
9 | 12 |
|
10 | | -## Tabular: `distill_predict` |
| 13 | +## Tabular: distill with `distill_predict`, serve with `predict` |
11 | 14 |
|
12 | 15 | ```sql |
| 16 | +-- once: fit and register the student |
13 | 17 | SELECT model_id, holdout_metric FROM distill_predict( |
14 | 18 | 'SELECT f1, f2, label FROM training', |
15 | | - '{"target":"label","student_kind":"gbt"}'); |
| 19 | + '{"target":"label","student_id":"churn-v1","student_kind":"gbt"}'); |
| 20 | + |
| 21 | +-- forever: serve it. train_query is NULL because the student already |
| 22 | +-- learned; only the rows to predict are needed. |
| 23 | +SELECT row_ref, prediction, confidence FROM predict(NULL, |
| 24 | + 'SELECT id, f1, f2 FROM customers', |
| 25 | + '{"model":"churn-v1"}'); |
16 | 26 | ``` |
17 | 27 |
|
18 | | -`student_kind` is `tree` (a single CART), `gbt` (a gradient-boosted forest with |
19 | | -second-order leaves, which matches or beats tuned XGBoost on most tasks), or |
20 | | -`mlp` (a one-hidden-layer net for boundaries a tree renders poorly). With `proba` |
21 | | -and `classes` the student learns the teacher's full probability distribution |
22 | | -(soft-label distillation), not just its argmax. The result registers a model you |
23 | | -call with [`predict()`](../../reference/functions/). |
| 28 | +The `NULL` first argument is the signature of serving a student: `predict` |
| 29 | +normally takes a training query for in-context models, but a student carries |
| 30 | +its training inside its blob, so passing one is an error. |
| 31 | + |
| 32 | +`student_kind` is `tree` (a single CART), `gbt` (a gradient-boosted forest |
| 33 | +with second-order leaves, which matches or beats tuned XGBoost on most |
| 34 | +tasks), or `mlp` (a one-hidden-layer net for boundaries a tree renders |
| 35 | +poorly). With `proba` and `classes` the student learns the teacher's full |
| 36 | +probability distribution (soft-label distillation), not just its argmax. |
24 | 37 |
|
25 | | -## Forecasting: `distill_forecast` |
| 38 | +## Forecasting: distill with `distill_forecast`, serve with `forecast` |
26 | 39 |
|
27 | 40 | ```sql |
| 41 | +-- once: an onnx teacher labels sliding windows in-database (needs the |
| 42 | +-- opt-in onnx build; window mode below needs nothing extra) |
28 | 43 | SELECT model_id, train_rmse FROM distill_forecast( |
29 | 44 | 'SELECT series_key, value FROM history ORDER BY series_key, t', |
30 | 45 | '{"teacher":"chronos-onnx","context":48,"horizon":12,"student_id":"fc"}'); |
| 46 | + |
| 47 | +-- forever: serve it through the ordinary aggregate |
| 48 | +SELECT forecast(ts, value, 12, '{"model":"fc"}') FROM readings; |
| 49 | + |
| 50 | +-- or let auto decide: a registered student competes against the |
| 51 | +-- statistical baselines automatically, and the document's "model" |
| 52 | +-- field tells you when it won |
| 53 | +SELECT forecast(ts, value, 12, '{"model":"auto"}') FROM readings; |
31 | 54 | ``` |
32 | 55 |
|
33 | | -The student is a DLinear/TiDE-style net (a linear skip plus a small residual). It |
34 | | -serves through [`forecast()`](../../reference/functions/) like any other model, competes under `'{"model":"auto"}'` automatically the moment it is registered, and |
35 | | -can compete in [`auto`](../auto-and-conformal/) selection. |
| 56 | +The student is a DLinear/TiDE-style net (a linear skip plus a small |
| 57 | +residual). Without an onnx teacher, `distill_forecast` also trains directly |
| 58 | +from windowed rows: each training row is `context` window columns followed by |
| 59 | +`horizon` continuation columns (your own teacher's forecasts, computed |
| 60 | +anywhere and stored as columns). |
36 | 61 |
|
37 | 62 | ## Distribute the student |
38 | 63 |
|
39 | 64 | A distilled student is a small native row in `_predict_models` (kilobytes, |
40 | | -`runtime='tree'`). It serves in the zero-dependency build with no ONNX runtime, |
41 | | -travels with a snapshot or fork of the database, and can be copied to another |
42 | | -database and used there. So you can **distill once on a capable machine and |
43 | | -predict everywhere SQLite runs**. Only distilling *from a live foundation-model |
44 | | -teacher* needs the optional ONNX build; a student distilled from its own target |
45 | | -columns needs nothing extra. |
| 65 | +`runtime='tree'`). It serves in the zero-dependency build with no ONNX |
| 66 | +runtime, travels with a snapshot or fork of the database, and can be copied |
| 67 | +to another database and used there. So you can **distill once on a capable |
| 68 | +machine and predict everywhere SQLite runs**. Only distilling *from a live |
| 69 | +foundation-model teacher* needs the optional ONNX build; a student distilled |
| 70 | +from its own target columns needs nothing extra. |
0 commit comments