Skip to content
This repository was archived by the owner on Aug 12, 2026. It is now read-only.

Commit 8038572

Browse files
committed
0017: host imports, after a review ran every claim and changed four of five
The first draft was written from what the API looks like it should do. The review ran it. Each finding verified here before folding in. E would have aborted the JVM. `{:wasi true}` -> add_wasip2 is not enough: add_wasip2 succeeds, instantiate succeeds, and the first WASI call panics in crates/c-api/src/store.rs:105 -- "called Option::unwrap() on a None value" -- then "fatal runtime error: failed to initiate panic, aborting". rc=134. Adding wasmtime_context_set_wasi(cx, wasi_config_new()) makes the same binary print a number. So :wasi carries a config, which is a design surface of its own and one whose default is a security decision -- inherit_env would leak the host environment into the guest. Deny-by-default. B's evidence was false and one command contradicted it. The draft said wasmtime's missing-import message has "no name" in it. It names the interface and the function; the draft quoted only the innermost line of a chain ok! already prints in full. Same shape as 0007. The host-side check is also unimplementable alongside WASI, because reflection cannot tell which imports add_wasip2 satisfies -- so missing imports are wasmtime's to report, and only the extra-import half survives. C was asserted symmetric because both directions use the same word. Ownership runs the opposite way: wasmtime frees what the callback writes, measured by returning static storage and getting POINTER_BEING_FREED_WAS_NOT_ALLOCATED inside add_func's closure. host.clj's lower-fn allocates every payload from an Arena, so reusing it in that direction is heap corruption. It needs an allocator parameter. D was half right. The error does reach the caller with its message -- and then the store is poisoned: the next call traps without entering the callback, and a new instance in the same store fails the same way. So an import that throws must mark the instance dead, or the user sees two identical-looking ex-infos that differ in whether the handle still works. And resource imports were filed as a falsifier rather than as work. Every wasi:io interface hands out a handle, so a map keyed iface#func cannot express what they need. That is inside this unit. Claude-Session: https://claude.ai/code/session_01XF5Hfq4Ca2N2XYEzWQQuHt
1 parent 3ccf845 commit 8038572

2 files changed

Lines changed: 165 additions & 85 deletions

File tree

‎doc/design/0017-host-imports.md‎

Lines changed: 156 additions & 76 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,23 @@
11
# 0017 — Host imports: letting a component call Clojure
22

3-
**Status:** proposed · 2026-07-30 · the mechanism is verified (`bb
4-
spike-import`); this is the API over it, and no code exists
3+
**Status:** proposed · 2026-07-30 · rewritten after an adversarial review ran
4+
every claim. Four of the first draft's five decisions changed, one of them
5+
because the draft would have aborted the JVM.
56

67
## The question
78

8-
`cljwit.host` can call a component. It cannot yet be called *by* one, which
9-
means it refuses every component that imports anything — including all of WASI.
10-
`0014` named this its most-likely-to-fire falsifier.
9+
`cljwit.host` can call a component. It cannot yet be called *by* one, so it
10+
refuses every component that imports anything — including all of WASI.
1111

1212
What is settled, measured 2026-07-30:
1313

1414
- **An FFM upcall stub reaches Clojure from inside a component call.** `bb
15-
spike-import`: `run(20) = 41`, the host saw `[20]`. Pure Clojure, bound to a
16-
`reify` — no bytecode generation and no C shim, the same answer `0011`
17-
reached for the outbound direction.
18-
- **A component's imports reflect exactly like its exports** — interface,
19-
parameter names, types, result — through
15+
spike-import`: `run(20) = 41`. Pure Clojure, bound to a `reify`.
16+
- **Imports reflect exactly like exports**, through
2017
`wasmtime_component_type_import_count`/`_nth`.
21-
- **A host callback cannot re-enter the instance that is executing**: `wasm
22-
trap: cannot enter component instance`. `0014` D and E survive.
23-
- **`wasmtime_component_linker_add_wasip2` supplies all of WASI 0.2 in one
24-
call**, so hand-writing `wasi:cli` is not the price of entry.
25-
- **`wasmtime_error_new(const char *)` exists**, so a Clojure exception has
26-
somewhere to go.
18+
- **A host callback cannot re-enter the executing instance**: `wasm trap:
19+
cannot enter component instance`. `0014` D and E survive.
20+
- **`wasmtime_error_new` exists**, so a Clojure exception has somewhere to go.
2721

2822
## The decision
2923

@@ -33,91 +27,177 @@ What is settled, measured 2026-07-30:
3327
(host/instantiate art {:imports {"local:imp/host@0.1.0#twice" (fn [v] (* 2 v))}})
3428
```
3529

36-
The identity is the exact WIT name, `iface#func`, exactly as `0014` B decided
37-
for exports. One naming rule for both directions.
30+
The identity is the exact WIT name, `iface#func`, as `0014` B decided. One
31+
naming rule for both directions.
3832

39-
### B. The host reflects what the component needs and refuses a mismatch, both ways
33+
**The stubs should not be per-instance, even though the functions are.**
34+
`Linker.upcallStub` measures **~7 µs warm and 30–160 µs cold**, against a 20 µs
35+
instantiate; a fresh closure per request means a fresh stub and a fresh linker
36+
per request. The shape that avoids it — one stub per import *name*, cached on
37+
the artifact, dispatching through the store's data pointer
38+
(`wasmtime_store_new(engine, data, finalizer)` / `wasmtime_context_get_data`,
39+
both present in 47.0.1) to the per-instance Clojure function — keeps A's
40+
per-request state and makes stubs and linker engine-scoped. **Unimplemented and
41+
unverified**; recorded because the naive shape has a measured cost.
4042

41-
A missing import throws at `instantiate`, **naming it**. wasmtime's own message
42-
is `function implementation is missing` with no name, which is a worse version
43-
of the `nil`-on-typo failure `0014` already rejected.
43+
### B. The host checks for *extra* imports, and lets wasmtime report missing ones
4444

45-
An import supplied but not needed throws too. A typo silently ignored is the
46-
same failure wearing the other hat.
45+
An import supplied but not needed throws at `instantiate`, naming it. A typo
46+
silently ignored is the `nil`-on-typo failure `0014` rejected.
4747

48-
### C. Marshalling is the same compiled lift and lower, run in the opposite direction
48+
**Missing imports are wasmtime's to report.** The first draft claimed its
49+
message was `function implementation is missing` "with no name". That was
50+
false — it quotes only the innermost line of a chain `ok!` already reads:
4951

50-
An import's parameters are *lifted* into Clojure and its result *lowered* back
51-
— the mirror of an export. `lift-fn` and `lower-fn` already compile from a
52-
reflected type tree; nothing new is needed but the direction.
52+
```
53+
linker_instantiate: component imports instance `local:imp/host@0.1.0`,
54+
but a matching implementation was not found in the linker
55+
Caused by:
56+
0: instance export `twice` has the wrong type
57+
1: function implementation is missing
58+
```
59+
60+
Interface and function are both named. And a host-side check is not merely
61+
redundant, it is **unimplementable** alongside E: reflection reports WASI
62+
imports identically to user imports, so with WASI on, every WASI import would
63+
look missing, and wasmtime does not expose the set `add_wasip2` defines. A
64+
`wasi:` prefix rule is not a substitute — a component can import
65+
`wasi:notreal/thing@0.2.0`, which passes any prefix rule and fails anyway.
66+
67+
### C. Marshalling is **not** the export direction run backwards
68+
69+
An import's arguments are lifted the same way — the argument buffer is
70+
wasmtime's and is invalid after the call, exactly as `0014` E found for
71+
results, so eager lifting carries over.
72+
73+
**The result side does not.** wasmtime takes ownership of what the callback
74+
writes and **frees it**. Measured: returning a `malloc`'d buffer works;
75+
returning a pointer into static storage aborts with
76+
`POINTER_BEING_FREED_WAS_NOT_ALLOCATED` inside
77+
`wasmtime_component_linker_instance_add_func`'s closure.
5378

54-
### D. A Clojure exception becomes a `wasmtime_error_t *`, and never unwinds into native frames
79+
`host.clj`'s `lower-fn` allocates **every** payload from a
80+
`java.lang.foreign.Arena` — string bytes, list/tuple/record element buffers,
81+
flags name vectors, and the boxed vals behind `option`/`result`/`variant`.
82+
Handing wasmtime an Arena pointer to `free()` is heap corruption. The import
83+
direction needs `malloc` and `wasmtime_component_val_new`, so `lower-fn` needs
84+
an allocator parameter rather than a hard-wired arena.
5585

56-
The callback catches `Throwable`, converts it with `wasmtime_error_new`, and
57-
returns it. A JVM exception propagating through a native frame is undefined
58-
behaviour, and this project has already lost a JVM to a non-unwinding panic
59-
(`0012`'s surrogate `char`).
86+
This inverts `0014` E as well: there the host must never `val_delete` a result;
87+
here the host must never *retain* one.
6088

61-
### E. WASI is a flag, not a map
89+
### D. A Clojure exception becomes a `wasmtime_error_t *` — and kills the instance
90+
91+
The callback catches `Throwable`, converts with `wasmtime_error_new`, returns
92+
it. A JVM exception unwinding through native frames is fatal: measured, it
93+
prints the Clojure stack trace and then `Unrecoverable uncaught exception
94+
encountered. The VM will now exit`.
95+
96+
**But the designed path is not an ordinary error.** Measured: the message does
97+
reach the caller, and then the store is poisoned — the next call gets `wasm
98+
trap: cannot enter component instance` without entering the callback, and a
99+
*new instance in the same store* fails the same way.
100+
101+
So `cljwit.host` must **mark the instance dead** when an import throws, and say
102+
so. Otherwise a user sees two `ex-info`s that look alike — a WIT `result` err,
103+
which leaves the instance usable, and a host-import throw, which does not — and
104+
every later call reports a trap with the cause gone. `0016` recorded exactly
105+
that failure shape.
106+
107+
The `catch` handler must also be incapable of throwing: `(ex-message e)` can be
108+
`nil`, and the C string for `wasmtime_error_new` needs an arena alive inside
109+
the callback.
110+
111+
### E. WASI is a config, not a flag
62112

63113
```clojure
64-
(host/instantiate art {:wasi true})
114+
(host/instantiate art {:wasi {}}) ;; deny-by-default
65115
```
66116

67-
calls `add_wasip2`. Supplying a hundred functions by hand to run `wasi:cli` is
68-
not an API, and the C API already refuses to make you.
117+
`add_wasip2` alone **aborts the process**. Measured: it succeeds, `instantiate`
118+
succeeds, and the first WASI call panics in
119+
`crates/c-api/src/store.rs:105` — `called Option::unwrap() on a None value` —
120+
then `fatal runtime error: failed to initiate panic, aborting`. Adding
121+
`wasmtime_context_set_wasi(cx, wasi_config_new())` makes the same binary print
122+
a number.
123+
124+
So `:wasi` must carry a `wasi_config_t`, which is a design surface of its own —
125+
argv, env, stdio, preopens, `inherit_network`, `allow_ip_name_lookup`. **The
126+
default is a security decision**: `inherit_env` would leak the host
127+
environment into the guest. Deny-by-default, and every capability named
128+
explicitly.
69129

70-
## Why
130+
The config is consumed by `set_wasi` even on error, so it is per-instantiate,
131+
not per-artifact. `add_wasip2` costs **0.033–0.037 ms**, against `0014` C's
132+
0.02 ms instantiate — so turning WASI on nearly triples instantiation.
71133

72-
**A, because two naming rules would be one too many.** `0014` B paid for the
73-
exact-WIT-string identity with measurements about keywords; imports do not get
74-
to relitigate it.
134+
### F. Resource imports are inside this unit, not after it
75135

76-
**B, because instantiation is the only moment the host knows both sides.** It
77-
has the component's requirements from reflection and the user's map in hand. A
78-
mismatch discovered later is discovered by wasmtime, in a message with no name
79-
in it.
136+
A host interface that hands out a handle — which is every one of `wasi:io`'s —
137+
declares a `resource` item that a map keyed `iface#func` cannot express.
138+
Measured: `instantiate` fails with `instance export 'token' has the wrong type
139+
/ resource implementation is missing`. It needs
140+
`wasmtime_component_linker_instance_add_resource` with a destructor callback (a
141+
second upcall shape) and `wasmtime_component_resource_host_new` to mint
142+
handles.
80143

81-
**D, because the failure mode is the worst one available.** Every other error
82-
in this library is an `ex-info`. An exception crossing into wasmtime's frames
83-
is a crash with no stack trace, and the cost of preventing it is a `try`.
144+
The first draft listed this as a falsifier. It is not one waiting to fire.
84145

85146
## Alternatives rejected
86147

87-
- **Imports on the artifact rather than the instance.** The linker is
88-
engine-scoped, so this would be possible — and it would make every instance
89-
share one set of host functions, which is wrong the moment two requests want
90-
different state. `0014` C put per-request state on the instance.
91-
- **A separate `link` step** between `compile` and `instantiate`. Honest about
92-
where the linker lives, and a fourth lifetime for something that has no
93-
independent lifetime: nothing can use a linker except to instantiate.
94-
- **Letting a missing import fail lazily**, at the first call. Cheaper to
95-
implement and it moves the error away from the mistake.
96-
- **Requiring an explicit `wasi:cli` map even when `add_wasip2` exists.**
97-
Consistent, and unusable.
148+
- **Imports on the artifact, sharing one set of host functions.** Wrong the
149+
moment two requests want different state. But see A: the *stubs* can be
150+
artifact-scoped without the *state* being, through the store data pointer,
151+
and the first draft rejected artifact scope on the assumption that they
152+
could not.
153+
- **A separate `link` step.** A fourth lifetime for something with no
154+
independent lifetime.
155+
- **Letting a missing import fail lazily**, at the first call. Moves the error
156+
away from the mistake — and is now moot, since wasmtime reports it at
157+
instantiate with both names.
158+
- **A host-side missing-import check.** The first draft's B. Removed: its
159+
stated justification was false, and it cannot coexist with E.
160+
- **`{:wasi true}`.** The first draft's E. It aborts the process.
98161
- **Passing the raw `wasmtime_component_val_t` to the Clojure function.**
99162
Fastest, and it makes every import a chance to corrupt memory.
100163

101164
## What would falsify this
102165

103-
- **An import that is not a function.** A component may import a *type* or a
104-
*resource*; `wasmtime_component_item_t` has kinds for both. A: the map is
105-
keyed by function name and has nowhere to put them.
106-
- **The upcall stub's lifetime.** It is bound to an `Arena`, and it must
107-
outlive every call. If the instance's arena is the wrong scope — say the stub
108-
must outlive the *linker* rather than the instance — A is in the wrong place.
109-
- **`add_wasip2` needing a WASI context on the store.**
110-
`wasmtime_context_set_wasi` exists and takes a config; whether `add_wasip2`
111-
works without one, and what E should do about the config, is unverified.
112-
- **Whether a returned error surfaces to the outer caller as an error rather
113-
than a trap that poisons the instance.** D assumes it does. Unmeasured.
114-
- **The cost.** An upcall is not free and nothing here has been timed. If a
115-
guest calls a host import in a loop, the number that matters is per-upcall,
116-
and `0013`'s discipline applies: predict, then measure.
166+
- **The store-data-pointer shape in A.** Unimplemented. If the data pointer is
167+
already spoken for, or dispatch through it costs more than the 7 µs it saves,
168+
A stays naive.
169+
- **Marking the instance dead being wrong** if some import errors turn out not
170+
to poison the store. Measured for one shape; not for a trap raised inside the
171+
callback versus an error returned from it.
172+
- **The `malloc` requirement in C.** Measured through one string result. Lists,
173+
records and nested vals are assumed to follow; `wasmtime_component_val_new`'s
174+
"the val provided is taken" wording says they should.
175+
- **A deny-by-default WASI config being unusable** — if the common case needs
176+
five capabilities named every time, the shape is wrong even if the default is
177+
right.
178+
- **The upcall cost.** ~7 µs warm is measured; per-call overhead is not, and
179+
`0013`'s discipline applies before anyone optimises it.
180+
181+
## What the first draft got wrong
182+
183+
Recorded as `0012`, `0013` and `0016` were.
184+
185+
- **E would have aborted the JVM** on the first WASI call, in a shape the note
186+
presented as the convenient one.
187+
- **B's evidence was false**, and one command contradicted it. Quoting the
188+
innermost line of an error chain the library already prints in full is the
189+
same shape as `0007`.
190+
- **C was asserted symmetric because both directions use the same word.**
191+
Ownership runs the opposite way, and the existing `lower-fn` would have
192+
handed wasmtime Arena pointers to `free`.
193+
- **F was filed as a falsifier** rather than as work, which would have made
194+
every `wasi:io`-shaped interface fail after the design was declared done.
195+
196+
The pattern: four of the five rest on what an API *looks like it should do*.
197+
The review ran them.
117198

118199
## Resources
119200

120201
- `dev/resources/imp.{wit,wat}` and `bb spike-import` — the mechanism.
121-
- `0014` — the naming rule, the lifetimes, and the concurrency check this
122-
inherits.
123-
- `0016` — the other direction of the same marshalling.
202+
- `0014` — the naming rule, lifetimes and concurrency check this inherits.
203+
- `0016` — the other direction, and the poisoning failure shape D must avoid.

‎doc/status.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -17,15 +17,15 @@ _Short by design, and printed at every session start — so findings live in
1717
express today. What is left of `0012` is `map`, `list<T,N>`,
1818
`stream`/`future` and `error-context`, none of which are in a shipped
1919
release.
20-
2. **Host imports.** The mechanism works — an FFM upcall stub reaches Clojure
21-
from inside a component call (`bb spike-import`), pure Clojure, no C shim.
22-
What is left is the `cljwit.host` API for declaring them. A component's
23-
imports reflect exactly like its exports — interface, parameter names,
24-
types — and `wasmtime_component_linker_add_wasip2` supplies all of WASI 0.2
25-
in one call, so hand-writing `wasi:cli` is not the price of entry. `0014` D and E
26-
both **survive**: a host callback cannot re-enter the instance that is
27-
executing (`wasm trap: cannot enter component instance`), so the nested
28-
call that would clobber the argument buffer cannot happen.
20+
2. **Host imports (`0017`).** The mechanism works — an FFM upcall stub reaches
21+
Clojure from inside a component call (`bb spike-import`), pure Clojure, no C
22+
shim. The API is designed and argued, and an adversarial review changed four
23+
of its five decisions. Three findings that outlive the note: `add_wasip2`
24+
without `wasmtime_context_set_wasi` **aborts the process** at the first WASI
25+
call; an import that throws poisons the whole store, not just the call; and
26+
wasmtime **frees** what the callback writes, so `lower-fn`'s Arena
27+
allocation cannot be reused in that direction. Resource imports are inside
28+
this unit, not after it — every `wasi:io` interface needs one.
2929
3. **`0012`'s `ex-data` contract.** It promises a WIT type name that reflection
3030
cannot supply; `0015` declined to be the codegen layer that would, so the
3131
contract has to shrink instead.

0 commit comments

Comments
 (0)