Skip to content

Commit f9865a6

Browse files
mstrathmanclaude
andcommitted
docs: reframe distillation around teachers you may actually ship
The tabular FM ecosystem has shifted to non-commercial licensing (TabFM; TabPFN 2.5/2.6/3), so leading with 'distill a foundation model' promised something a commercial user often cannot legally do. The distillation story now ranks teachers by shippability: your own labels or model first (no license question; compressing your production pipeline into the database is the common case), permissively licensed FMs second (Chronos, TabPFN-2), evaluation-only frontier models last (benchmark zero-shot, buy a license to distill). Section renamed 'Distill a model into your database'; guide lead, homepage cards, and the GitHub description follow. tabarena-full.md states its TabFM campaign is a license-permitted evaluation of the mechanism and points at tabpfn.md for the same mechanism under a permissive teacher, and its stale distill() name is fixed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent a1cb053 commit f9865a6

4 files changed

Lines changed: 46 additions & 21 deletions

File tree

‎README.md‎

Lines changed: 26 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,9 @@ SELECT r.* FROM forecast_rows((SELECT forecast(ts, value, 24)
2929
```
3030

3131
None of this ships your data to a cloud model. It runs in-process, on CPU, in
32-
microseconds. And where a foundation model would be too heavy to call per
33-
query, you [**distill it once into a tiny native student**](#distill-a-foundation-model-into-your-database)
32+
microseconds. And when a stronger model (your own trained model, or a
33+
foundation model you are licensed to distill) is too heavy to call per query,
34+
you [**distill it once into a tiny native student**](#distill-a-model-into-your-database)
3435
that lives inside your database and runs anywhere.
3536

3637
## Is it accurate?
@@ -68,22 +69,36 @@ All of it runs on CPU, in-process, with no network and no GPU.
6869
> **Pre-alpha.** The API and the SQL surface are
6970
> unstable and may change without notice. Not yet recommended for production.
7071
71-
## Distill a foundation model into your database
72+
## Distill a model into your database
7273

73-
Foundation models are the accuracy ceiling and the deployment problem: too
74-
slow to call per query on CPU, too heavy to ship inside an app. Distillation
75-
is sqlite-predict's answer, and it is a SQL primitive like everything else:
76-
run the big model once as a teacher, compress what it learned into a native
77-
student a few kilobytes big, and serve that student in microseconds from the
78-
zero-dependency core.
74+
A strong model is usually a deployment problem: your production pipeline
75+
needs Python and a container, and a foundation model is too slow to call per
76+
query and too heavy to ship inside an app. Distillation is sqlite-predict's
77+
answer, and it is a SQL primitive like everything else: run the teacher once,
78+
compress what it learned into a native student a few kilobytes big, and serve
79+
that student in microseconds from the zero-dependency core.
80+
81+
Any teacher works, and the common ones rank by what you may ship:
82+
83+
1. **Your own labels or your own model's predictions**: the default path,
84+
with no license question at all. Compress the XGBoost or sklearn pipeline
85+
you already trust into a student that serves inside SQLite, no Python at
86+
serve time.
87+
2. **Permissively licensed foundation models**: Chronos (Apache-2.0) for
88+
time series, TabPFN-2 (Prior Labs License, distillation expressly
89+
permitted with attribution) for tabular.
90+
3. **Evaluation-only frontier models** (TabFM, TabPFN-3): benchmark them
91+
zero-shot to see what accuracy is on the table; distilling them for
92+
commercial use requires a license from their vendor.
7993

8094
The student is not a file on the side. **It is a row in your database**, so it
8195
snapshots, forks, branches, and syncs with the data it predicts, and it
8296
travels wherever the database file goes. An agent that distills a model owns
8397
that model the same way it owns its tables.
8498

8599
```sql
86-
-- once: distill (your labels, or a teacher's precomputed predictions)
100+
-- once: distill. The target column holds your labels, or your existing
101+
-- model's predictions (which compresses that model into the database).
87102
SELECT model_id, holdout_metric FROM distill_predict(
88103
'SELECT tenure, spend, churned FROM customers',
89104
'{"target":"churned","student_id":"churn-v1","student_kind":"gbt"}');
@@ -207,7 +222,7 @@ By default `distill_predict()` trains directly on the target column. That
207222
column can hold your labels, or a strong teacher's predictions computed
208223
offline: run the teacher once over your training rows on a GPU box, store
209224
what it predicts, and distill compresses it into a student that runs anywhere
210-
([example above](#distill-a-foundation-model-into-your-database)).
225+
([example above](#distill-a-model-into-your-database)).
211226
Pass a `teacher` to instead relabel the rows with a registered model first
212227
(for example, `'{"teacher":"knn5-incontext", ...}'` compresses the in-context
213228
k-NN into a standalone tree).

‎benchmarks/results/tabarena-full.md‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,12 @@
22

33
This runs the whole [TabArena-v0.1 suite](https://tabarena.ai) (OpenML suite
44
457, 51 datasets, 38 classification and 13 regression) through the extension's
5-
own `distill()` and `predict()`, with the real TabFM as a distillation teacher.
5+
own `distill_predict()` and `predict()`, with the real TabFM as a distillation
6+
teacher. This campaign is an evaluation of the distillation mechanism, run
7+
under TabFM's non-commercial license (which permits benchmarking); distilling
8+
TabFM for commercial use requires a license from Google. The same mechanism
9+
with a permissively licensed teacher is measured in
10+
[`tabpfn.md`](tabpfn.md).
611
See [`datasets.md`](datasets.md) for what each dataset predicts.
712

813
It is not TabArena's official protocol. That uses full data, nested

‎website/src/content/docs/guides/distillation.md‎

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,16 @@ title: Distillation
33
description: Compress a slow teacher into a tiny native student, then serve it like any other model.
44
---
55

6-
Foundation models are accurate but too heavy to call per query on CPU. The
7-
answer is distillation: run the teacher once to label your data, then fit a
8-
small **native student** that runs in the zero-dependency core with no
9-
runtime. Every distilled student registers under the `student_id` you give
10-
it, and you serve it by passing that id as the `model` option of the normal
11-
serving call. There is no separate "serve a student" API.
6+
A strong teacher is usually too heavy to call per query: your production
7+
model needs its Python stack, and a foundation model needs seconds of CPU or
8+
a GPU. The answer is distillation: run the teacher once to label your data,
9+
then fit a small **native student** that runs in the zero-dependency core
10+
with no runtime. The teacher can be your own model (put its predictions in
11+
the target column and you have compressed it into the database), a
12+
permissively licensed foundation model, or your plain labels. Every distilled
13+
student registers under the `student_id` you give it, and you serve it by
14+
passing that id as the `model` option of the normal serving call. There is no
15+
separate "serve a student" API.
1216

1317
## Tabular: distill with `distill_predict`, serve with `predict`
1418

‎website/src/content/docs/index.mdx‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ SELECT forecast(ts, value, 24) FROM readings;
3131
-- compose, so ORMs and query builders work unchanged.
3232
SELECT city, forecast(ts, value, 24) FROM readings GROUP BY city;
3333

34-
-- and when a foundation model is too heavy to call per query:
34+
-- and when a stronger model is too heavy to call per query:
3535
-- distill it once into a few-KB native student that lives in your
3636
-- database, then serve the student in microseconds
3737
SELECT model_id FROM distill_forecast('SELECT * FROM windows',
@@ -47,8 +47,9 @@ SELECT forecast(ts, value, 24, '{"model":"traffic-v1"}') FROM readings;
4747
Pure C99. No ONNX runtime, no GPU, no network at serve time.
4848
</Card>
4949
<Card title="Distill once, serve forever" icon="seti:shell">
50-
Compress a foundation model into a few-KB student that lives in your
51-
database and snapshots, forks, and syncs with it. See [Distillation](./guides/distillation/).
50+
Compress your own model, or a foundation model you are licensed to
51+
distill, into a few-KB student that lives in your database and
52+
snapshots, forks, and syncs with it. See [Distillation](./guides/distillation/).
5253
</Card>
5354
<Card title="Measured, reproducible numbers" icon="bars">
5455
~0.89 MASE distilled forecasting, SOTA-level anomaly detection, calibrated intervals. See [Benchmarks](./benchmarks/).

0 commit comments

Comments
 (0)