Skip to content

Commit b5e9997

Browse files
committed
feat(worker): install native Miller in hosted image
1 parent af65312 commit b5e9997

5 files changed

Lines changed: 18 additions & 1 deletion

File tree

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,7 @@ Current qualification note: Lighthouse is green, browser p95 is `377.26 ms` for
6767

6868
- Render Blueprint: `render.yaml`
6969
- Render worker deploy notes: `docs/deploy/RENDER_WORKER.md`
70+
- Worker container: `apps/worker/Dockerfile` now installs native Miller for hosted fallback paths
7071

7172
## License
7273

apps/worker/Dockerfile

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
1-
FROM node:22-alpine
1+
FROM node:22-bookworm-slim
2+
3+
RUN apt-get update \
4+
&& apt-get install -y --no-install-recommends miller ca-certificates \
5+
&& rm -rf /var/lib/apt/lists/*
26

37
WORKDIR /app
48

@@ -8,6 +12,12 @@ RUN npm install
812
COPY apps/worker/src ./src
913
COPY apps/worker/tsconfig.json ./tsconfig.json
1014

15+
ENV NODE_ENV=production
16+
ENV PORT=8797
17+
ENV WORKER_ARTIFACT_TTL_SECONDS=900
18+
1119
EXPOSE 8797
1220

21+
HEALTHCHECK --interval=15s --timeout=5s --start-period=10s --retries=5 CMD node -e "fetch('http://127.0.0.1:' + (process.env.PORT || 8797) + '/health').then((r)=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))"
22+
1323
CMD ["npx", "tsx", "src/index.ts"]

docs/deploy/RENDER_WORKER.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ This repo includes a `render.yaml` Blueprint for the native worker.
1212
- Health check path: `/health`
1313
- Runtime port: `10000`
1414
- Worker artifact TTL: `900`
15+
- Native Miller: installed in the container via `apps/worker/Dockerfile`
1516

1617
## Deploy flow
1718

@@ -26,4 +27,5 @@ This repo includes a `render.yaml` Blueprint for the native worker.
2627

2728
- The worker is modeled as a web service because it exposes an HTTP API at `/v1/run`.
2829
- Local compose still uses port `8797`; Render uses `PORT=10000`.
30+
- The worker image now installs the `mlr` binary so hosted health checks can expose real native Miller availability.
2931
- Public hosted verification is not complete until the deployed Render URL is recorded and checked.

docs/qc/APPENDIX_B_REPORT.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,11 +44,13 @@ Acceptance evidence:
4444
Blocked:
4545
- 21.0 Hosted web URL now exists and returned `200` at `https://chayprabs.github.io/csv-jsonl-miller/`, and the latest `main` runs for `CI`, `Deploy Pages`, and `Publish Worker Image` all succeeded on commit `8936ed5`.
4646
The repo now also contains a Render Blueprint in `render.yaml` and deploy notes in `docs/deploy/RENDER_WORKER.md` for creating a public worker service.
47+
The worker container now installs native Miller directly in `apps/worker/Dockerfile`.
4748
A hosted worker URL is still not provisioned, so this item remains blocked.
4849
- 21.1 Browser Miller-WASM integration is still blocked.
4950
Browser DuckDB-WASM is now exercised by `pnpm --filter @csvshape/web smoke:duckdb`, which runs supported DuckDB-WASM chains over the ecommerce CSV, access-log JSONL, and wide-sales CSV samples and records the results in `docs/qc/benchmarks/browser-duckdb-smoke.json`.
5051
Worker-native DuckDB is packaged via `@duckdb/node-api`, reports `duckdbNative: true` on `/health`, and executes inline CSV/JSONL SQL plus Parquet export in local tests.
5152
Worker-native Miller now has a repeatable smoke run via `pnpm --filter @csvshape/worker smoke:mlr`, which returns `engine=mlr-native`, `rowCount=3`, and CSV output for the paid-order ecommerce subset in `docs/qc/benchmarks/native-mlr-smoke.json`.
53+
The worker container now installs the `mlr` binary directly, which makes the hosted native Miller path concrete at the image level even though the browser Miller-WASM path is still unresolved.
5254
`pnpm probe:miller-wasm` now records a direct upstream `GOOS=js GOARCH=wasm` probe in `docs/qc/benchmarks/browser-miller-wasm-probe.json`; at the current Miller version, Go fails the build in the generated parser with `function too big ... exceeds 65536 blocks`, so there is still no usable browser Miller artifact to verify.
5355
- 21.13 Privacy evidence is now local-only rather than hosted.
5456
Browser-first privacy evidence is now available both locally and on the hosted app.

docs/qc/SECTION_21_STATUS.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,12 @@ It is not yet a qualified release record.
3434
The static app is now hosted at `https://chayprabs.github.io/csv-jsonl-miller/`, and a direct HTTP check returned `200`.
3535
GitHub Actions on `main` now show green `CI`, `Deploy Pages`, and `Publish Worker Image` runs for commit `8936ed5`.
3636
The repo now also includes a Render Blueprint in `render.yaml` plus deploy notes in `docs/deploy/RENDER_WORKER.md` for a public worker URL.
37+
The worker container now installs native Miller in `apps/worker/Dockerfile`, so the hosted image path aligns with the advertised native fallback engine set.
3738
A hosted worker URL is still missing, so this gate is not fully closed yet.
3839
- [ ] 21.1 Real Miller-WASM, DuckDB-WASM, native Miller, and native DuckDB integration evidence.
3940
Browser DuckDB-WASM is exercised locally across the acceptance samples via `pnpm --filter @csvshape/web smoke:duckdb`, with artifact output in `docs/qc/benchmarks/browser-duckdb-smoke.json`.
4041
Worker-native DuckDB execution is covered by local tests, and worker-native Miller now has a repeatable local smoke run via `pnpm --filter @csvshape/worker smoke:mlr` with artifact output in `docs/qc/benchmarks/native-mlr-smoke.json`.
42+
The worker container now explicitly installs the `mlr` binary in `apps/worker/Dockerfile`, strengthening the hosted native Miller path.
4143
`pnpm probe:miller-wasm` now records a repeatable upstream browser probe in `docs/qc/benchmarks/browser-miller-wasm-probe.json`; the current direct `GOOS=js GOARCH=wasm` build fails inside Miller's generated parser with `function too big ... exceeds 65536 blocks`.
4244
Browser Miller-WASM integration evidence is still missing because that upstream build path does not currently produce a usable browser artifact.
4345
- [ ] 21.12 Performance evidence for browser p95 and worker throughput.

0 commit comments

Comments
 (0)