Skip to content

Commit fa0917b

Browse files
committed
feat(nrepl): ADR-0170 — re-architect the nREPL server to base-protocol CIDER fidelity
Smell-audited: 3: ADR-0170 accepted with the mandatory DA fork; the DA's Cycle-budget-defer catch (shared eval engine NOW, not deferred) adopted as Alt 2; bencode readString borrow fixed to honor the module's own arena-owned contract instead of patching around it in the framer. Empirically-confirmed defects fixed (two independent replay probes): - read-loop stranding (fillMore blocked with complete dicts buffered — pipelined CIDER requests stranded off-by-one; the REPL-RET killer) - 4KiB frame ceiling (larger messages reset the connection; C-c C-k killer) - session identity (every clone returned the same id; replies stamped the connection constant instead of echoing the request session) - error replies (raw Zig error name; err bundled with done, double-done, eval continued past a failing form) - missing completions/complete/lookup/info/eldoc; describe drift + stale hardcoded version Shape (src/app/nrepl/ package + shared engine): - nrepl/transport.zig — drain-all-buffered framing, growable recv buffer, responseFor session/id echo choke point (bb idiom) - nrepl/session.zig — distinct per-clone ids, per-session *1..*e (pinned, gc_rooting.md D6) + current-ns isolation - nrepl/ops.zig — comptime op table; describe DERIVES ops + reports build_options.version; babashka 3-message error protocol with the CLI's caret rendering - app/eval_session.zig — shared REPL engine: the CLI REPL gains *1/*2/*3/*e in the same cycle (F-011); *1..*e interned in core.clj (upstream shape) - runtime/introspect.zig — completion candidates + var-meta lookup shared by line_editor TAB and the nREPL ops Verified: e2e CIDER-replay (11 new assertions) green; real nrepl/nrepl 1.3.1 Java client end-to-end (eval / *1 / completions / lookup arglists / describe / eval-error); CLI REPL *1 manually; 1210 unit tests; zlinter. D-117/D-118 re-narrowed by code-truth (the 2026-07-07 "CIDER ops LANDED" claim was false — only load-file had landed).
1 parent ac50600 commit fa0917b

15 files changed

Lines changed: 1821 additions & 457 deletions

File tree

‎.dev/debt.yaml‎

Lines changed: 31 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -3161,20 +3161,38 @@ discharged:
31613161
resolution: |-
31623162
docs | **cljw man-page rendering**: Row 14.13 (D-066) shipped `docs/spec/error_format.md` (Markdown). v5 §13.2 + §24.2 also wanted a manpage-style render for `man cljw`. Discharge: write `docs/man/cljw.1` in mdoc(7) format covering the subcommand surface (eval / repl / nrepl / build / render-error) + env vars (CLJW_ERROR_FORMAT / CLJW_ERROR_LOG) + a synopsis section. Estimated ~150 lines of mdoc; install path under a future packaging cycle. | Opportunistic — packaging / distro polish phase | 2026-05-28
31633163
- id: "D-118"
3164-
discharged_at: "Phase 15+ — when `*out*` / `*err*` dynamic-binding lands"
3165-
resolution: |-
3166-
runtime | **nREPL stdout/stderr capture**: Row 14.10 (2026-05-28) landed `cljw nrepl` with empty `out` / `err` keys in eval responses — the cw v1 print primitive writes to the process stdout, not a per-session buffer. CIDER's REPL buffer relies on `out` responses to display side-effect prints; today `(println "x")` runs at the server's stdout, invisible to the connected client. Discharge: bind `*out*` / `*err*` dynamic Vars to a per-session captured Writer that pushes each chunk as a separate nREPL `{out | err}` response. Estimated ~120 LOC + dynamic-binding integration. Independent of D-117 (multi-session / CIDER ops). | Phase 15+ when `*out*` / `*err*` binding lands | 2026-05-28
3164+
discharged_at: "RE-NARROWED 2026-07-13 (ADR-0170): out AND err both captured + streamed; residual = per-WRITE streaming (today per top-level form)"
3165+
resolution: |-
3166+
RE-NARROWED 2026-07-13 (ADR-0170): `*out*` AND `*err*` are both
3167+
bound to capture writers per eval (eval_session.zig) and streamed
3168+
to the client as `out` / `err` responses per top-level form.
3169+
Residual divergence FROM babashka (which streams per WRITE via a
3170+
proxy Writer, mid-eval): a long-running form's prints arrive only
3171+
when the form returns — cosmetic for fast single-threaded evals;
3172+
the fix shape is a callback-writer Value that emits on each
3173+
`.write` (text_io), taken up if a user reports the delay.
3174+
ORIGINAL: runtime | **nREPL stdout/stderr capture**: Row 14.10 (2026-05-28) landed `cljw nrepl` with empty `out` / `err` keys in eval responses — the cw v1 print primitive writes to the process stdout, not a per-session buffer. CIDER's REPL buffer relies on `out` responses to display side-effect prints; today `(println "x")` runs at the server's stdout, invisible to the connected client. Discharge: bind `*out*` / `*err*` dynamic Vars to a per-session captured Writer that pushes each chunk as a separate nREPL `{out | err}` response. Estimated ~120 LOC + dynamic-binding integration. Independent of D-117 (multi-session / CIDER ops). | Phase 15+ when `*out*` / `*err*` binding lands | 2026-05-28
31673175
- id: "D-117"
3168-
discharged_at: "Phase 15+ — when concurrency primitives enable multi-session (RE-NARROWED 2026-07-07: only (a) multi-session remains)"
3169-
resolution: |-
3170-
RE-NARROWED BY CODE-TRUTH 2026-07-07 (D-536 reconciliation): of the
3171-
five deferred items, (b) interrupt, (c) load-file + CIDER ops, and
3172-
(d) --port 0 auto-assign are LANDED (the nrepl e2e runs eval / out /
3173-
load-file / interrupt / ls-sessions green; auto-port fixed 2026-07-07
3174-
via Zig 0.16 Socket.address — feature_deps#runtime/nrepl/auto_port =
3175-
landed). Remaining: (a) multi-session thread-per-session (the D-418
3176-
concurrency area) + (e) middleware only if Tier-A tooling demands.
3177-
ORIGINAL:
3176+
discharged_at: "Concurrency gap area — when thread-per-session lands (RE-NARROWED 2026-07-13 by ADR-0170: only true mid-eval interrupt + stdin remain)"
3177+
resolution: |-
3178+
RE-NARROWED 2026-07-13 (ADR-0170 re-architecture): the 2026-07-07
3179+
"CIDER ops LANDED" claim was FALSE — completions/complete/lookup/
3180+
info/eldoc did not exist (only load-file had landed; the
3181+
verify_actual_pattern_not_proxy lesson). ADR-0170 landed them for
3182+
real, plus distinct per-clone sessions with per-session *1..*e +
3183+
current-ns, request-session echo, drain-before-block framing,
3184+
>4KiB frames, the babashka 3-message error protocol with
3185+
CLI-grade rendering, and describe derived from the op table
3186+
(e2e: phase14_nrepl.sh case 4). Remaining, genuinely
3187+
concurrency-gated: (a) thread-per-session → true mid-eval
3188+
`interrupt` (today acks done+session-idle — an eval owns the only
3189+
thread) + `stdin` op (read-line over the wire). (e) middleware
3190+
framework stays not-wanted (bb precedent: op handlers suffice).
3191+
RE-NARROWED-2026-07-07 TEXT (superseded): of the five deferred
3192+
items, (b) interrupt, (c) load-file + CIDER ops, and (d) --port 0
3193+
auto-assign are LANDED (... auto-port fixed 2026-07-07 via Zig
3194+
0.16 Socket.address — feature_deps#runtime/nrepl/auto_port =
3195+
landed). ORIGINAL:
31783196
tooling | **nREPL polish — multi-session, interrupt, CIDER ops, auto-port**: Row 14.10 (2026-05-28) landed a minimal 4-op nREPL server (clone / describe / eval / close) on single concurrent session. Deferred items: (a) multi-session via thread-per-session (needs Phase 15+ `std.Thread.spawn`); (b) `interrupt` op (single-thread eval cannot interrupt itself); (c) CIDER-specific ops `complete` / `info` / `load-file` / `eldoc`; (d) `--port 0` auto-assign needs `getsockname` surface (Zig 0.16 `Server` doesn't expose bound addr — see `runtime/nrepl/auto_port` feature_deps entry); (e) middleware framework if Tier-A tooling demands it. Estimated ~400-600 LOC total. | Phase 15+ — concurrency activation OR a CIDER-tooling test corpus needs a missing op | 2026-05-28
31793197
- id: "D-116"
31803198
discharged_at: "Opportunistic — when an interactive REPL session needs arrow-key history"

‎.dev/gc_rooting.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,7 @@ Fixed by bracketing the alloc in the ADR-0150 fabrication no-collect region
157157
| D3 | `agent.zig` `gc.pin(agent_val)` | Agent (drainer thread) |
158158
| D4 | `trackHeap` → `heap_objects` + `gc.registerPersistentMark` → `persistent_marks: ArrayList(*anyopaque)` | process-lifetime gpa objects (`Function`, `ProtocolDescriptor`, `ProtocolFn`, `TypeDescriptorRef`) OUTSIDE `gc.allocations`; `collect` clears their mark bit each cycle so their trace re-runs (ADR-0095 Class 1) |
159159
| D5 | `trackHeap` callers | `tree_walk.zig` (Function ×3), `protocol.zig` (×3), `type_descriptor.zig` (×1) |
160+
| D6 | `eval_session.StarState` `gc.pin`/`unpin` (ADR-0170) | REPL star history `*1`/`*2`/`*3`/`*e` held BETWEEN evals — CLI REPL (one StarState per session) + every nREPL session (`nrepl/session.zig` Registry). Pin on rotate/setE, unpin the displaced value; `release()` on session close / REPL exit. Nothing else references a REPL result between evals — an unpinned `*1` would be swept and the next deref touches freed memory |
160161

161162
**Migration-impact:** D1-D3 are addressable `ArrayList(Value)` — rewrite each
162163
entry on move. **D4 is the single highest-risk migration site:** raw

‎.dev/structure_plan.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -111,9 +111,13 @@ ClojureWasm/
111111
├─ main.zig Layer 3 entry (shrinks in Phase 8 ★split)
112112
├─ app/ ★new (Phase 8 entry, D-031)
113113
│ ├─ repl.zig
114+
│ ├─ repl/ line_editor
115+
│ ├─ eval_session.zig shared REPL eval engine (ADR-0170; CLI + nREPL)
114116
│ ├─ runner.zig file / -e / stdin 共通の eval runner
115117
│ ├─ self_host_loader.zig Phase 8
116-
│ ├─ nrepl_server.zig Phase 10
118+
│ ├─ nrepl.zig + nrepl/ transport / session / ops (ADR-0170 package;
119+
│ │ supersedes the single-file nrepl_server.zig
120+
│ │ reservation + runtime/cljw/repl/NReplServer.zig)
117121
│ ├─ builder.zig Phase 12 (cljw build)
118122
│ └─ pod_runner.zig Phase 16 (zwasm v2)
119123
├─ runtime/ Layer 0

‎src/app/eval_session.zig‎

Lines changed: 262 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,262 @@
1+
// SPDX-License-Identifier: EPL-2.0
2+
//! Shared REPL evaluation engine (ADR-0170): the per-form
3+
//! read → analyze → eval loop with `*1`/`*2`/`*3`/`*e` history
4+
//! rotation, optional `*out*`/`*err*` capture, stop-on-first-error,
5+
//! and CLI-grade error rendering — driven through caller-supplied
6+
//! sinks. Consumers: the nREPL `eval`/`load-file` ops (bencode
7+
//! sinks) and the CLI REPL (terminal sinks). One engine, so the two
8+
//! REPLs cannot drift (F-011): `(inc *1)` works identically in
9+
//! CIDER and `cljw repl`, and both render the same caret error text.
10+
//!
11+
//! ns handling is deliberately NOT here: the CLI REPL wants `in-ns`
12+
//! to persist globally, the nREPL op wants per-session save/restore —
13+
//! each caller manages `env.current_ns` around this call.
14+
15+
const std = @import("std");
16+
const Writer = std.Io.Writer;
17+
18+
const Runtime = @import("../runtime/runtime.zig").Runtime;
19+
const Env = @import("../runtime/env.zig").Env;
20+
const env_mod = @import("../runtime/env.zig");
21+
const Value = @import("../runtime/value/value.zig").Value;
22+
const GcHeap = @import("../runtime/gc/gc_heap.zig").GcHeap;
23+
const Reader = @import("../eval/reader.zig").Reader;
24+
const analyzeForm = @import("../eval/analyzer/analyzer.zig").analyze;
25+
const macro_dispatch = @import("../eval/macro_dispatch.zig");
26+
const driver = @import("../eval/driver.zig");
27+
const root_set = @import("../runtime/gc/root_set.zig");
28+
const print = @import("../runtime/print.zig");
29+
const text_io = @import("../runtime/io/text_io.zig");
30+
const dispatch = @import("../runtime/dispatch.zig");
31+
const error_render = @import("error_render.zig");
32+
33+
/// REPL history state (`*1` `*2` `*3` `*e`) with the GC pin
34+
/// discipline built in.
35+
///
36+
/// GC-ROOT: held star Values are `GcHeap.pin`ned (permanent_roots)
37+
/// while stored and unpinned on overwrite / release — between evals
38+
/// nothing else references a REPL result, so an unpinned `*1` would
39+
/// be swept and the next deref would touch freed memory. See
40+
/// `.dev/gc_rooting.md` § REPL star history.
41+
pub const StarState = struct {
42+
gc: *GcHeap,
43+
values: [4]Value = @splat(.nil_val),
44+
45+
pub const idx_1 = 0;
46+
pub const idx_2 = 1;
47+
pub const idx_3 = 2;
48+
pub const idx_e = 3;
49+
50+
pub fn init(gc: *GcHeap) StarState {
51+
return .{ .gc = gc };
52+
}
53+
54+
/// Roll the history chain with a fresh eval result: `*3` drops
55+
/// (unpin), `*2`→`*3`, `*1`→`*2`, `v`→`*1` (pin).
56+
pub fn rotate(self: *StarState, v: Value) !void {
57+
try self.gc.pin(v);
58+
_ = self.gc.unpin(self.values[idx_3]);
59+
self.values[idx_3] = self.values[idx_2];
60+
self.values[idx_2] = self.values[idx_1];
61+
self.values[idx_1] = v;
62+
}
63+
64+
/// Set `*e` (most recent caught exception value).
65+
pub fn setE(self: *StarState, v: Value) !void {
66+
try self.gc.pin(v);
67+
_ = self.gc.unpin(self.values[idx_e]);
68+
self.values[idx_e] = v;
69+
}
70+
71+
/// Unpin everything (owner teardown).
72+
pub fn release(self: *StarState) void {
73+
for (self.values) |v| _ = self.gc.unpin(v);
74+
self.values = @splat(.nil_val);
75+
}
76+
};
77+
78+
pub const Options = struct {
79+
source: []const u8,
80+
/// Error-rendering label: `<repl:N>`, `<nrepl>`, or a file name.
81+
source_label: []const u8,
82+
/// History state to bind + rotate, or null to skip `*1`..`*e`.
83+
stars: ?*StarState = null,
84+
/// Bind `*out*`/`*err*` to capture writers and emit the captured
85+
/// text through the sink per top-level form (nREPL). False = let
86+
/// prints flow to the process stdio (CLI REPL).
87+
capture_output: bool = false,
88+
/// true → emit a value per top-level form (REPL eval);
89+
/// false → emit only the LAST form's value (clj load-file).
90+
emit_each_value: bool = true,
91+
};
92+
93+
/// Evaluate `opts.source` form by form. The sink is duck-typed:
94+
/// onValue(text) — a form's printed result
95+
/// onOut(text) / onErrOut(text) — captured *out* / *err*
96+
/// onError(rendered, err_name, thrown) — rich rendered error text +
97+
/// the Zig error name + the
98+
/// thrown Value when present
99+
/// Returns true when every form evaluated; false when it stopped on
100+
/// the first error (JVM parity — later forms never run).
101+
pub fn evalSource(
102+
rt: *Runtime,
103+
env: *Env,
104+
macro_table: *const macro_dispatch.Table,
105+
arena: std.mem.Allocator,
106+
scratch: std.mem.Allocator,
107+
opts: Options,
108+
sink: anytype,
109+
) !bool {
110+
const core_ns = env.findNs("clojure.core");
111+
112+
// One frame for the whole call: star bindings (so `(inc *1)`
113+
// resolves) + the capture writers. Values roll per form via
114+
// setBinding on this same frame.
115+
var frame: env_mod.BindingFrame = .{};
116+
var v1: ?*env_mod.Var = null;
117+
var v2: ?*env_mod.Var = null;
118+
var v3: ?*env_mod.Var = null;
119+
var ve: ?*env_mod.Var = null;
120+
var out_var: ?*env_mod.Var = null;
121+
var err_var: ?*env_mod.Var = null;
122+
if (core_ns) |core| {
123+
if (opts.stars) |stars| {
124+
v1 = core.resolve("*1");
125+
v2 = core.resolve("*2");
126+
v3 = core.resolve("*3");
127+
ve = core.resolve("*e");
128+
if (v1) |v| try frame.bindings.put(scratch, v, stars.values[StarState.idx_1]);
129+
if (v2) |v| try frame.bindings.put(scratch, v, stars.values[StarState.idx_2]);
130+
if (v3) |v| try frame.bindings.put(scratch, v, stars.values[StarState.idx_3]);
131+
if (ve) |v| try frame.bindings.put(scratch, v, stars.values[StarState.idx_e]);
132+
}
133+
if (opts.capture_output) {
134+
out_var = core.resolve("*out*");
135+
err_var = core.resolve("*err*");
136+
if (out_var) |v| try frame.bindings.put(scratch, v, try text_io.mintStringWriter(rt));
137+
if (err_var) |v| try frame.bindings.put(scratch, v, try text_io.mintStringWriter(rt));
138+
}
139+
}
140+
env_mod.pushFrame(&frame);
141+
defer env_mod.popFrame();
142+
143+
var reader = Reader.init(arena, opts.source);
144+
var last_value: ?[]const u8 = null;
145+
var ok = true;
146+
147+
while (true) {
148+
const form_opt = reader.read() catch |err| {
149+
try flushCaptured(out_var, err_var, sink);
150+
try renderAndEmit(scratch, opts, err, sink);
151+
ok = false;
152+
break;
153+
};
154+
const form = form_opt orelse break;
155+
156+
// D-430: per-form analysis bracket (roots literals through eval).
157+
var af: root_set.AnalysisFrame = undefined;
158+
root_set.beginAnalysis(&af, rt.gc.infra);
159+
defer root_set.endAnalysisPersist(&af, &rt.gc);
160+
161+
const node = analyzeForm(arena, rt, env, null, form, macro_table) catch |err| {
162+
try flushCaptured(out_var, err_var, sink);
163+
try handleEvalError(opts, ve, err, scratch, sink);
164+
ok = false;
165+
break;
166+
};
167+
var locals: [driver.MAX_LOCALS]Value = [_]Value{.nil_val} ** driver.MAX_LOCALS;
168+
const result = driver.evalForm(rt, env, &locals, arena, node) catch |err| {
169+
try flushCaptured(out_var, err_var, sink);
170+
try handleEvalError(opts, ve, err, scratch, sink);
171+
ok = false;
172+
break;
173+
};
174+
try flushCaptured(out_var, err_var, sink);
175+
try rearmCapture(rt, out_var, err_var);
176+
177+
// Rotate history + refresh the live frame so the NEXT form's
178+
// `*1` sees this result.
179+
if (opts.stars) |stars| {
180+
try stars.rotate(result);
181+
if (v1) |v| _ = env_mod.setBinding(v, stars.values[StarState.idx_1]);
182+
if (v2) |v| _ = env_mod.setBinding(v, stars.values[StarState.idx_2]);
183+
if (v3) |v| _ = env_mod.setBinding(v, stars.values[StarState.idx_3]);
184+
}
185+
186+
var aw: Writer.Allocating = .init(scratch);
187+
try print.printResult(rt, env, &aw.writer, result);
188+
const value_str = aw.written();
189+
if (opts.emit_each_value) {
190+
try sink.onValue(value_str);
191+
} else {
192+
last_value = value_str;
193+
}
194+
}
195+
if (ok and !opts.emit_each_value) {
196+
if (last_value) |lv| try sink.onValue(lv);
197+
}
198+
return ok;
199+
}
200+
201+
/// Emit any captured `*out*` / `*err*` text, then re-arm both capture
202+
/// writers for the next form (a fresh string writer per flush keeps
203+
/// each `out` response scoped to its form).
204+
fn flushCaptured(out_var: ?*env_mod.Var, err_var: ?*env_mod.Var, sink: anytype) !void {
205+
if (out_var) |v| {
206+
if (env_mod.findBinding(v)) |w| {
207+
const captured = text_io.writerBytes(w);
208+
if (captured.len > 0) try sink.onOut(captured);
209+
}
210+
}
211+
if (err_var) |v| {
212+
if (env_mod.findBinding(v)) |w| {
213+
const captured = text_io.writerBytes(w);
214+
if (captured.len > 0) try sink.onErrOut(captured);
215+
}
216+
}
217+
}
218+
219+
/// Re-arm the capture writers after a flush. Split from
220+
/// `flushCaptured` because re-minting needs `rt`, which the reader-
221+
/// error path doesn't have on its unwind.
222+
fn rearmCapture(rt: *Runtime, out_var: ?*env_mod.Var, err_var: ?*env_mod.Var) !void {
223+
if (out_var) |v| _ = env_mod.setBinding(v, try text_io.mintStringWriter(rt));
224+
if (err_var) |v| _ = env_mod.setBinding(v, try text_io.mintStringWriter(rt));
225+
}
226+
227+
fn handleEvalError(
228+
opts: Options,
229+
ve: ?*env_mod.Var,
230+
err: anyerror,
231+
scratch: std.mem.Allocator,
232+
sink: anytype,
233+
) !void {
234+
// Peek the thrown Value BEFORE rendering — renderError consumes the
235+
// threadlocal throw state.
236+
const thrown = dispatch.last_thrown_exception;
237+
if (thrown) |t| {
238+
if (opts.stars) |stars| {
239+
try stars.setE(t);
240+
if (ve) |v| _ = env_mod.setBinding(v, t);
241+
}
242+
}
243+
try renderAndEmitThrown(scratch, opts, err, thrown, sink);
244+
}
245+
246+
fn renderAndEmit(scratch: std.mem.Allocator, opts: Options, err: anyerror, sink: anytype) !void {
247+
try renderAndEmitThrown(scratch, opts, err, null, sink);
248+
}
249+
250+
fn renderAndEmitThrown(scratch: std.mem.Allocator, opts: Options, err: anyerror, thrown: ?Value, sink: anytype) !void {
251+
var aw: Writer.Allocating = .init(scratch);
252+
error_render.renderError(&aw.writer, .{ .file = opts.source_label, .text = opts.source }, err) catch {};
253+
try sink.onError(aw.written(), @errorName(err), thrown);
254+
}
255+
256+
// eval_session has no direct unit tests: its contract is pinned by the
257+
// nREPL e2e (phase14_nrepl.sh case 4) + the REPL e2e, and StarState's
258+
// pin discipline is unit-tested from session.zig's registry tests.
259+
// The `test` block below keeps the module in the discovery graph.
260+
test {
261+
std.testing.refAllDecls(StarState);
262+
}

0 commit comments

Comments
 (0)