Skip to content

Commit 3a3c61c

Browse files
authored
Merge pull request #17 from jgarzik/updates
JIT
2 parents 705d7e0 + 773a2da commit 3a3c61c

21 files changed

Lines changed: 1172 additions & 283 deletions

File tree

‎.github/workflows/TestingCI.yml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,13 @@ jobs:
1818
run: cargo build --release --verbose
1919
- name: Run tests
2020
run: cargo test --release --verbose
21+
# The integration suite again, every program run by `xbasic64 run`
22+
# (compiled into memory, in the compiler's process) instead of as an
23+
# executable: the two must agree.
24+
- name: Run tests through the JIT
25+
run: cargo test --release --test integration
26+
env:
27+
XB_JIT: "1"
2128
- name: Run fmt check
2229
run: cargo fmt --all -- --check
2330
- name: Run clippy
@@ -33,6 +40,7 @@ jobs:
3340
echo "== $f"
3441
./target/release/xbasic64 "$f" -o /tmp/example
3542
/tmp/example </dev/null >/dev/null
43+
./target/release/xbasic64 run "$f" </dev/null >/dev/null
3644
done
3745
3846
linux-aarch64:
@@ -48,6 +56,13 @@ jobs:
4856
run: cargo build --release --verbose
4957
- name: Run tests
5058
run: cargo test --release --verbose
59+
# The integration suite again, every program run by `xbasic64 run`
60+
# (compiled into memory, in the compiler's process) instead of as an
61+
# executable: the two must agree.
62+
- name: Run tests through the JIT
63+
run: cargo test --release --test integration
64+
env:
65+
XB_JIT: "1"
5166
- name: Run fmt check
5267
run: cargo fmt --all -- --check
5368
- name: Run clippy
@@ -63,6 +78,7 @@ jobs:
6378
echo "== $f"
6479
./target/release/xbasic64 "$f" -o /tmp/example
6580
/tmp/example </dev/null >/dev/null
81+
./target/release/xbasic64 run "$f" </dev/null >/dev/null
6682
done
6783
6884
macos-arm64:
@@ -78,6 +94,13 @@ jobs:
7894
run: cargo build --release --verbose
7995
- name: Run tests
8096
run: cargo test --release --verbose
97+
# The integration suite again, every program run by `xbasic64 run`
98+
# (compiled into memory, in the compiler's process) instead of as an
99+
# executable: the two must agree.
100+
- name: Run tests through the JIT
101+
run: cargo test --release --test integration
102+
env:
103+
XB_JIT: "1"
81104
- name: Run fmt check
82105
run: cargo fmt --all -- --check
83106
- name: Run clippy
@@ -93,6 +116,7 @@ jobs:
93116
echo "== $f"
94117
./target/release/xbasic64 "$f" -o /tmp/example
95118
/tmp/example </dev/null >/dev/null
119+
./target/release/xbasic64 run "$f" </dev/null >/dev/null
96120
done
97121
98122
windows-native:
@@ -113,6 +137,13 @@ jobs:
113137
run: cargo build --release --verbose
114138
- name: Run tests
115139
run: cargo test --release --verbose
140+
# The integration suite again, every program run by `xbasic64 run`
141+
# (compiled into memory, in the compiler's process) instead of as an
142+
# executable: the two must agree.
143+
- name: Run tests through the JIT
144+
run: cargo test --release --test integration
145+
env:
146+
XB_JIT: "1"
116147
- name: Run fmt check
117148
run: cargo fmt --all -- --check
118149
- name: Run clippy
@@ -146,6 +177,11 @@ jobs:
146177
# [uint32] then threw on the very value it was there to show.
147178
throw ("run failed: {0} (exit {1}, 0x{2:X8})" -f $_.Name, $LASTEXITCODE, $LASTEXITCODE)
148179
}
180+
# The same program through the JIT: `xbasic64 run`.
181+
& ./target/release/xbasic64.exe run $_.FullName | Out-Null
182+
if ($LASTEXITCODE -ne 0) {
183+
throw ("xbasic64 run failed: {0} (exit {1}, 0x{2:X8})" -f $_.Name, $LASTEXITCODE, $LASTEXITCODE)
184+
}
149185
}
150186
151187
# An example is a whole program, so a crash in one names the program and

‎CLAUDE.md‎

Lines changed: 28 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,10 @@ cargo test strings::
2323
cargo run -- program.bas # Output: program, beside the source
2424
cargo run -- program.bas -o out # Custom output name
2525
cargo run -- -S program.bas # Cranelift IR to <output>.clif instead (no object, no link)
26+
cargo run -- run program.bas # Compile into memory and run it at once (the JIT)
27+
28+
# The integration suite with every program run by `xbasic64 run`
29+
XB_JIT=1 cargo test --test integration
2630
```
2731

2832
## Architecture
@@ -58,8 +62,14 @@ Source → Lexer → Parser → Sema → CodeGen → Object → Executable
5862
Every call goes through `FnGen::call`, which builds the call from it; tests
5963
read a call graph from `rt/src` and check that `fallible` is exactly "can
6064
reach `raise_with_code`" and `fresh` exactly "can reach `alloc_str`"
65+
- **jit.rs** - `xbasic64 run`: the same lowering (`codegen::lower`) into a
66+
cranelift-jit `JITModule`, then main is called in-process. The runtime is
67+
the `rt` crate linked into the compiler (feature `host`); `symbols()` gives
68+
the JIT the address of every helper, runtime word and message, and C
69+
function generated code names, since an executable exports none by name
6170
- **main.rs** - CLI driver: reads source, runs the pipeline, writes the object
62-
and the runtime's, and links them with `cc` (or `link.exe`)
71+
and the runtime's, and links them with `cc` (or `link.exe`); or, for the
72+
`run` subcommand, hands the program to `jit.rs`
6373
- **build.rs** - Builds the runtime (`rt/`) with rustc for the target when the
6474
compiler is built: a `no_std` staticlib with fat LTO, of which only the
6575
crate's one merged object is kept (`$OUT_DIR/rt.o`; the rest of the
@@ -87,14 +97,18 @@ covers the workspace). In the runtime:
8797
Integration tests organized by feature area:
8898
- `integration.rs` - The one test crate: every module directory below is
8999
declared there, so a new one must be added
90-
- `common/mod.rs` - Test harness. `compile_and_run()` for the usual case;
100+
- `common/mod.rs` - Test harness. `XB_JIT=1` runs every program through
101+
`xbasic64 run` instead of as an executable (CI runs the suite both ways).
102+
`compile_and_run()` for the usual case;
91103
`compile_only()` for "this must be rejected, with this message";
92104
`compile_and_run_raw()` for "printed this, then aborted with this exit code";
93105
and `compile_to_clif()` / `uses_symbol()` for the IR the compiler generates
94106
- `docs/mod.rs` - Compiles every ```basic example in LANGREF.md and README.md
95107
- `megatest/` - One large program (`mega.bas`) exercising most of the language
96108
at once, so that features are tested coexisting rather than in isolation. It
97109
is its own assertion harness: a regression names itself
110+
- `jit/` - What is particular to `xbasic64 run`: how the process ends, what
111+
reaches stdout and stderr, and that it agrees with an executable
98112
- `codegen/` - Assertions on the Cranelift IR the compiler generates (through
99113
`common::compile_to_clif()`), and behavioural tests of code paths
100114
with special cases: constant divisors, scalars in SSA values, huge functions
@@ -124,10 +138,18 @@ Integration tests organized by feature area:
124138
a function gets no optimizer and the single-pass register allocator, since
125139
the backtracking one is superlinear with SSA values live across it; that
126140
code runs about 2x slower
127-
- **Every reference is colocated**: functions and data, the runtime's and libc's
128-
included, are reached PC-relative (`call rel32` and RIP-relative on x86-64,
129-
`bl` and adrp+add on aarch64); on Linux the link is `-no-pie`, and macOS,
130-
where executables are always PIE, needs nothing more
141+
- **Every reference is colocated in an executable**: functions and data, the
142+
runtime's and libc's included, are reached PC-relative (`call rel32` and
143+
RIP-relative on x86-64, `bl` and adrp+add on aarch64); on Linux the link is
144+
`-no-pie`, and macOS, where executables are always PIE, needs nothing more.
145+
`CodeGen::colocate` says so; the JIT clears it
146+
- **The JIT's code model** (`jit.rs`): the runtime and libc are the
147+
compiler's own, anywhere in its address space, so a reference to them is
148+
an absolute address (Cranelift's default for an import). The program's
149+
own code and data stay PC-relative, which needs them within 2 GB: they
150+
come from one arena, 64 MB, doubled -- and the program lowered again --
151+
when it runs out. A new runtime helper or C function needs an entry in
152+
`jit::symbols()`, which a test holds to rtsig
131153
- **Native only, two ISAs**: Cranelift's `x86` and `arm64` backends are built
132154
in, and the compiler targets the host it was built for. aarch64 Linux can be
133155
checked on an x86-64 host under qemu-user: cross-build with

0 commit comments

Comments
 (0)