Repository navigation
Commit 489e1bc
authored
refactor: make worker startup the only flow compilation path (#672)
# refactor: make worker startup the only flow compilation path
Implements #647 ahead of queue identity (#650) and per-step private queues (#651).
## Summary
- Every flow worker now compiles or verifies its imported flow definition before it tracks its worker function or inserts its worker row. A failed compilation registers nothing and mutates no `worker_functions` row.
- Removed public surfaces: `compileFlow()` and `compareFlowShapes()` from `@pgflow/dsl`, the ControlPlane edge function and its root export, the `pgflow compile` CLI command, the ControlPlane installer step, all `supabase/functions/pgflow/` fixtures, and the complete `FlowWorkerConfig.compilation` property (including `allowDataLoss`).
- SQL: `pgflow.ensure_flow_compiled(flow_slug, shape)` keeps two arguments only. The new Atlas migration explicitly drops `pgflow.ensure_flow_compiled(text, jsonb, boolean)`, so production can no longer authorize destructive recompilation.
- Local same-slug shape mismatch keeps destructive recompile (dev hot reload); production mismatch returns `mismatch` without touching data. Structural or unobserved semantic changes use a new concrete slug.
- Kept: `extractFlowShape()`, `FlowShape` types, SQL `_compare_flow_shapes()`, `_create_flow_from_shape()`, `create_flow()`, `add_step()`, `delete_flow_and_data()`.
- Docs rewritten for startup-only deployment: `/concepts/startup-compilation/` is canonical, legacy URLs redirect there, and the [pgflow 0.16.0 release notes](https://pgflow.dev/news/pgflow-0-16-0-startup-only-flow-compilation/) document the fenced breaking-upgrade procedure.
- Changeset bumps the fixed group (`@pgflow/core`, `@pgflow/dsl`, `@pgflow/client`, `@pgflow/edge-worker`, `pgflow`) as minor.
**Breaking:** the migration removes the old three-argument SQL signature — stop old workers before applying it, and upgrade the pgflow package set together. See "Remove manual flow compilation" in the update guide.
## Checks
All run locally on this branch:
- `pnpm nx verify-migrations core`, `pnpm nx gen-types core`, `pnpm nx verify-gen-types core --skip-nx-cache` — pass
- `pnpm nx test:pgtap core` — 1348/1348 pass (incl. new two-arg signature test)
- `pnpm nx run-many -t lint build --projects=cli,dsl,edge-worker,client` — pass
- `pnpm nx test cli` / `test dsl` (337) / `test client` — pass
- `pnpm nx test:unit edge-worker` (271) / `test:types edge-worker` / `test:integration edge-worker` (61/61) — pass
- `pnpm nx e2e cli` / `e2e edge-worker` (12/12) / `e2e client` (47/47) — pass
- `pnpm nx verify-exports edge-worker` / `client`, publish dry runs (npm/JSR) — pass
- `pnpm nx build website` — 133 pages, starlight-links-validator pass
- CLI help contains no `compile` command; packed exports contain no `ControlPlane`/`compileFlow`/`compareFlowShapes`
Closes #6471 parent 5118509 commit 489e1bc
91 files changed
Lines changed: 545 additions & 4683 deletions
File tree
- .changeset
- apps/demo/supabase/functions/pgflow
- pkgs
- client/__tests__/e2e
- cli
- __tests__
- commands
- compile
- install
- e2e
- scripts
- src
- commands
- compile
- install
- supabase/functions/pgflow
- core
- schemas
- src
- supabase
- migrations
- tests/ensure_flow_compiled
- dsl
- __tests__
- integration
- runtime
- src
- edge-worker
- src
- control-plane
- core
- flow
- supabase
- functions/pgflow
- tests
- e2e
- integration/flow
- types
- unit
- control-plane
- website
- src
- components
- content/docs
- build
- comparisons
- concepts
- deploy
- supabase
- get-started
- flows
- news
- reference
- configuration
- tutorials
- ai-web-scraper
- use-cases
- scripts
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
123 | 123 | | |
124 | 124 | | |
125 | 125 | | |
126 | | - | |
127 | | - | |
128 | | - | |
129 | | - | |
130 | | - | |
131 | | - | |
132 | | - | |
| 126 | + | |
133 | 127 | | |
134 | 128 | | |
135 | 129 | | |
| |||
203 | 197 | | |
204 | 198 | | |
205 | 199 | | |
206 | | - | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
207 | 204 | | |
208 | 205 | | |
209 | 206 | | |
| |||
281 | 278 | | |
282 | 279 | | |
283 | 280 | | |
284 | | - | |
285 | 281 | | |
286 | 282 | | |
287 | 283 | | |
| |||
330 | 326 | | |
331 | 327 | | |
332 | 328 | | |
333 | | - | |
| 329 | + | |
334 | 330 | | |
335 | 331 | | |
336 | 332 | | |
337 | 333 | | |
338 | | - | |
| 334 | + | |
| 335 | + | |
339 | 336 | | |
340 | 337 | | |
341 | 338 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
241 | 241 | | |
242 | 242 | | |
243 | 243 | | |
244 | | - | |
245 | 244 | | |
246 | 245 | | |
247 | 246 | | |
248 | 247 | | |
249 | | - | |
250 | 248 | | |
251 | 249 | | |
252 | 250 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
84 | 84 | | |
85 | 85 | | |
86 | 86 | | |
87 | | - | |
88 | | - | |
| 87 | + | |
| 88 | + | |
89 | 89 | | |
90 | 90 | | |
91 | 91 | | |
| |||
94 | 94 | | |
95 | 95 | | |
96 | 96 | | |
97 | | - | |
| 97 | + | |
98 | 98 | | |
99 | 99 | | |
100 | 100 | | |
| |||
This file was deleted.
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
30 | | - | |
| 30 | + | |
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
| 10 | + | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
14 | | - | |
| 13 | + | |
15 | 14 | | |
16 | 15 | | |
17 | 16 | | |
| |||
59 | 58 | | |
60 | 59 | | |
61 | 60 | | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
75 | | - | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | | - | |
80 | | - | |
81 | | - | |
82 | 61 | | |
83 | 62 | | |
84 | 63 | | |
| |||
91 | 70 | | |
92 | 71 | | |
93 | 72 | | |
94 | | - | |
95 | | - | |
96 | | - | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
0 commit comments