Skip to content

Commit 4c3b0f8

Browse files
committed
Update docs to reflect engine overview refactor and new recording
workflow.
1 parent 43134da commit 4c3b0f8

8 files changed

Lines changed: 140 additions & 131 deletions

File tree

Makefile

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,8 @@
33
PORT ?= 3000
44

55
.PHONY: serve serve-prod build install validate deploy-firebase \
6-
sql-definitions sql-docs check-sql-definitions check-window-aggregates
6+
sql-definitions sql-docs check-sql-definitions check-window-aggregates \
7+
recordings
78

89
serve:
910
@echo "Starting docs-site dev server on http://localhost:$(PORT)"
@@ -30,6 +31,14 @@ install:
3031
validate:
3132
@cd docs-site && npm run validate:docs
3233

34+
# Terminal GIFs used for REPL/TUI walkthroughs (opteryx-cli.md, upload-cli.md).
35+
# Each .tape is a VHS script driving the real, installed CLI - see
36+
# docs-site/scripts/recordings/README.md. Requires `brew install vhs`.
37+
recordings: ## Regenerate every terminal recording GIF from its .tape script
38+
@cd docs-site/scripts/recordings && for tape in *.tape; do \
39+
echo "-- $$tape"; vhs "$$tape" || exit 1; \
40+
done
41+
3342
# --- SQL reference generation ------------------------------------------------
3443
# The chain, first link to last. Nothing in it is hand-edited; to change what the
3544
# docs say about a function, edit the registrar in opteryx-core.

docs-site/content/docs/guides/upload-cli.md

Lines changed: 1 addition & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -160,30 +160,7 @@ Same flow, same calls. What it adds is that the plan stays on screen: at a
160160
scrolling prompt the table goes past once and correcting a type means retyping
161161
the whole command, and here the cursor moves down it.
162162

163-
```
164-
opteryx upload https://upload.opteryx.app
165-
166-
FILES
167-
part-0000.parquet 412.9 MB
168-
part-0001.parquet 398.1 MB
169-
170-
ACCOUNT
171-
acme-etl
172-
173-
TO
174-
acme.security.findings
175-
176-
PLAN a new dataset; these types were read from your data
177-
column sample type
178-
cve_id CVE-2026-00001 VARCHAR
179-
published 2026-08-02T04:22:07Z TIMESTAMP[us] was VARCHAR, converted
180-
› source_ip 10.1.7.13 IPV4 was VARCHAR, converted
181-
hosts 1 INT64
182-
score 0.5 FLOAT64 read and not written
183-
184-
these types were read from your data - nothing is written until you accept
185-
↑↓ column e retype x ignore ⏎ accept u upload h keys q quit
186-
```
163+
<img src="/images/cli/upload-tui.gif" alt="The opteryx-upload TUI: negotiating a contract for findings.csv against acme.security.findings, reviewing the plan (published cast to TIMESTAMP[us], source_ip cast to IPV4, score read and not written), then uploading and committing" width="1040">
187164

188165
`h` lists every key. `c` signs in, `a` browses for files, `t` sets the
189166
destination, `n` negotiates, `e` changes the type under the cursor, `x` drops a
-25.5 KB
Loading
1.9 KB
Loading

docs-site/public/images/engine-overview.svg

Lines changed: 110 additions & 103 deletions
Loading

docs-site/reference/internals/engine-overview.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ This document walks the path a query takes from SQL text to results.
1010

1111
The engine is a loop. SQL goes down one side, is progressively refined into a runnable plan, executes, and results come back up the other side.
1212

13-
![How Opteryx plans and runs a query: SQL is refined into a plan down the left and across the bottom, then the plan is refined until it runs going up the right, before results return to the user. Python stages are blue, native stages (Parser, Executor) are rust, and the Catalogue is green. The Physical Planner's handover to the Executor is the one boundary the engine crosses without calling back into Python.](/images/engine-overview.svg)
13+
![How Opteryx plans and runs a query: SQL is refined into a plan down the left and across the bottom, then the plan is refined until it runs going up the right, before results return to the user. Python stages are blue, native stages (Parser, Executor) are rust, and the Catalogue is green.](/images/engine-overview.svg)
1414

1515
The left column transforms *text*; the bottom row transforms a *plan*; the right column refines that plan until it can run. Each stage has a single, narrow responsibility.
1616

docs-site/scripts/recordings/README.md

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,12 @@ CLI; nothing in the GIFs is mocked up.
1212

1313
```bash
1414
brew install vhs # if you don't have it
15+
make recordings # from the repo root — regenerates every .tape here
16+
```
17+
18+
Or one at a time:
19+
20+
```bash
1521
cd docs-site/scripts/recordings
1622
vhs opteryx-repl.tape
1723
```
@@ -23,14 +29,24 @@ recording looks stale.
2329
Run from a directory outside `/tmp` — pyenv-shimmed `python -m <module>`
2430
invocations have been seen to fail there with an unrelated
2531
`No module named 'compression'` error caused by something in this machine's
26-
`/tmp`, not by Opteryx. `~` or the repo itself are fine.
32+
`/tmp`, not by Opteryx. `~` or the repo itself are fine. `make recordings`
33+
already runs from the repo root, so this only matters if you run `vhs` by hand.
2734

2835
If you use pyenv and `python -m opteryx` isn't on the version that has
2936
`opteryx-core` installed, set `PYENV_VERSION` before running `vhs`, e.g.
3037
`PYENV_VERSION=3.13.12 vhs opteryx-repl.tape`.
3138

39+
`upload-tui.tape` records against `mock_upload_server.py`, a small stand-in for
40+
the real upload service (see that file's docstring) — the tape starts and stops
41+
it itself, so nothing needs to be running first. It speaks the real
42+
`/v2/contracts` protocol and returns a fixed, scripted plan, so the real,
43+
installed `opteryx-upload` command runs unmodified. `acme` is a reserved
44+
workspace name kept for exactly this kind of example — use it for any workspace
45+
a recording or a doc example needs.
46+
3247
## Recordings
3348

3449
| Tape | Shows | Used in |
3550
|---|---|---|
3651
| `opteryx-repl.tape` | Starting the REPL, running a query, exiting | [The Opteryx Command Line](/docs/guides/opteryx-cli) |
52+
| `upload-tui.tape` | Negotiating, reviewing the plan, uploading and committing | [The Upload Command Line](/docs/guides/upload-cli) |

docs-site/scripts/recordings/upload-tui.tape

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ Set Shell "bash"
66
Set FontSize 15
77
Set FontFamily "Menlo"
88
Set Width 1040
9-
Set Height 640
9+
Set Height 480
1010
Set Padding 20
1111
Set Theme "Dracula"
1212
Set TypingSpeed 45ms

0 commit comments

Comments
 (0)