Skip to content

Commit 53927e2

Browse files
mstrathmanclaude
andcommitted
docs: show how to serve a distilled student, and fix a broken example
The distillation guide taught creation but never the serving call. It now pairs every distill with its serve: predict(NULL, apply, model=id) for tabular students, with the NULL-train signature explained (a student carries its training in its blob; passing a train query is an error), and forecast(ts, value, h, model=id) plus the auto pickup for forecast students. The tabular example was also broken as written: it omitted the required student_id and errored verbatim (verified against the engine before and after). The functions reference's predict() entry gains the student-serving sentence, and the duplicated auto-competes clause from an earlier edit is gone. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 02d38e7 commit 53927e2

2 files changed

Lines changed: 49 additions & 22 deletions

File tree

Lines changed: 46 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,45 +1,70 @@
11
---
22
title: Distillation
3-
description: Compress a slow teacher into a tiny native student that runs anywhere.
3+
description: Compress a slow teacher into a tiny native student, then serve it like any other model.
44
---
55

66
Foundation models are accurate but too heavy to call per query on CPU. The
7-
answer is distillation: run the teacher once to label your data, then fit a small
8-
**native student** that runs in the zero-dependency core with no runtime.
7+
answer is distillation: run the teacher once to label your data, then fit a
8+
small **native student** that runs in the zero-dependency core with no
9+
runtime. Every distilled student registers under the `student_id` you give
10+
it, and you serve it by passing that id as the `model` option of the normal
11+
serving call. There is no separate "serve a student" API.
912

10-
## Tabular: `distill_predict`
13+
## Tabular: distill with `distill_predict`, serve with `predict`
1114

1215
```sql
16+
-- once: fit and register the student
1317
SELECT model_id, holdout_metric FROM distill_predict(
1418
'SELECT f1, f2, label FROM training',
15-
'{"target":"label","student_kind":"gbt"}');
19+
'{"target":"label","student_id":"churn-v1","student_kind":"gbt"}');
20+
21+
-- forever: serve it. train_query is NULL because the student already
22+
-- learned; only the rows to predict are needed.
23+
SELECT row_ref, prediction, confidence FROM predict(NULL,
24+
'SELECT id, f1, f2 FROM customers',
25+
'{"model":"churn-v1"}');
1626
```
1727

18-
`student_kind` is `tree` (a single CART), `gbt` (a gradient-boosted forest with
19-
second-order leaves, which matches or beats tuned XGBoost on most tasks), or
20-
`mlp` (a one-hidden-layer net for boundaries a tree renders poorly). With `proba`
21-
and `classes` the student learns the teacher's full probability distribution
22-
(soft-label distillation), not just its argmax. The result registers a model you
23-
call with [`predict()`](../../reference/functions/).
28+
The `NULL` first argument is the signature of serving a student: `predict`
29+
normally takes a training query for in-context models, but a student carries
30+
its training inside its blob, so passing one is an error.
31+
32+
`student_kind` is `tree` (a single CART), `gbt` (a gradient-boosted forest
33+
with second-order leaves, which matches or beats tuned XGBoost on most
34+
tasks), or `mlp` (a one-hidden-layer net for boundaries a tree renders
35+
poorly). With `proba` and `classes` the student learns the teacher's full
36+
probability distribution (soft-label distillation), not just its argmax.
2437

25-
## Forecasting: `distill_forecast`
38+
## Forecasting: distill with `distill_forecast`, serve with `forecast`
2639

2740
```sql
41+
-- once: an onnx teacher labels sliding windows in-database (needs the
42+
-- opt-in onnx build; window mode below needs nothing extra)
2843
SELECT model_id, train_rmse FROM distill_forecast(
2944
'SELECT series_key, value FROM history ORDER BY series_key, t',
3045
'{"teacher":"chronos-onnx","context":48,"horizon":12,"student_id":"fc"}');
46+
47+
-- forever: serve it through the ordinary aggregate
48+
SELECT forecast(ts, value, 12, '{"model":"fc"}') FROM readings;
49+
50+
-- or let auto decide: a registered student competes against the
51+
-- statistical baselines automatically, and the document's "model"
52+
-- field tells you when it won
53+
SELECT forecast(ts, value, 12, '{"model":"auto"}') FROM readings;
3154
```
3255

33-
The student is a DLinear/TiDE-style net (a linear skip plus a small residual). It
34-
serves through [`forecast()`](../../reference/functions/) like any other model, competes under `'{"model":"auto"}'` automatically the moment it is registered, and
35-
can compete in [`auto`](../auto-and-conformal/) selection.
56+
The student is a DLinear/TiDE-style net (a linear skip plus a small
57+
residual). Without an onnx teacher, `distill_forecast` also trains directly
58+
from windowed rows: each training row is `context` window columns followed by
59+
`horizon` continuation columns (your own teacher's forecasts, computed
60+
anywhere and stored as columns).
3661

3762
## Distribute the student
3863

3964
A distilled student is a small native row in `_predict_models` (kilobytes,
40-
`runtime='tree'`). It serves in the zero-dependency build with no ONNX runtime,
41-
travels with a snapshot or fork of the database, and can be copied to another
42-
database and used there. So you can **distill once on a capable machine and
43-
predict everywhere SQLite runs**. Only distilling *from a live foundation-model
44-
teacher* needs the optional ONNX build; a student distilled from its own target
45-
columns needs nothing extra.
65+
`runtime='tree'`). It serves in the zero-dependency build with no ONNX
66+
runtime, travels with a snapshot or fork of the database, and can be copied
67+
to another database and used there. So you can **distill once on a capable
68+
machine and predict everywhere SQLite runs**. Only distilling *from a live
69+
foundation-model teacher* needs the optional ONNX build; a student distilled
70+
from its own target columns needs nothing extra.

‎website/src/content/docs/reference/functions.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,9 @@ of `(time, value)`, optionally with grouping columns. Columns: `series_key`,
7474

7575
Learns from `train_query` (features plus a `target` column) and predicts the
7676
target for every row of `apply_query`. Columns: `row_ref`, `prediction`,
77-
`confidence`, `status`.
77+
`confidence`, `status`. To serve a distilled student, pass its id as the
78+
`model` option and `NULL` as `train_query` (the student already learned):
79+
`predict(NULL, 'SELECT id, f1, f2 FROM t', '{"model":"churn-v1"}')`.
7880

7981
### `distill_predict(train_query [, options])`
8082

0 commit comments

Comments
 (0)