Skip to content

Commit 343582a

Browse files
committed
test-infra(vm): script-driven Windows VM leg for the local 3-OS gate
The Windows VM becomes a first-class, reproducible local-CI leg driven entirely over the local ssh link by maintained scripts: - provision-windows.sh: idempotent host-side provisioner — msys2 base, CLANGARM64 (native, fast) + CLANG64 (CI arch) toolchains, repo clone/update, full build, smoke check. One command recovers a fresh VM or a lost disk. - win.sh: daily driver — status/update/build/test/guards/smoke-install/ sh/push-file/asan-build. smoke-install runs the managed-install E2E with stderr visible, the class the CI smoke probe structurally hides. - windows-bootstrap.ps1: reduced to the one-time in-VM door-opener (OpenSSH + host key + GitHub-runner default-owner policy mirror), with the ISO delivery trick documented; toolchain work moved to the host-side provisioner. The previous winget path does not work over ssh and clipboard/shared folders do not exist on Windows-ARM guests. - README.md: full lifecycle — VM creation, clone-before-drivers rollback point, SPICE guest tools, bootstrap, provisioning, daily loop, and the honest limits (no ASan on aarch64-windows; ASan also faults under x64-on-ARM emulation, so Windows sanitizer coverage stays on CI). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent 5498af0 commit 343582a

4 files changed

Lines changed: 288 additions & 53 deletions

File tree

‎test-infrastructure/vm/README.md‎

Lines changed: 80 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -1,59 +1,105 @@
1-
# VM test legs (opt-in)
1+
# VM test legs — real-Windows and runner-macOS
22

33
Two failure classes cannot be reproduced by the container legs and need real
4-
virtual machines. Both stacks are free; the legs activate automatically in
5-
`run.sh` when the tooling and the named VM are detected, and print setup
6-
guidance otherwise.
4+
virtual machines. Both stacks are free. The Windows VM is a first-class part
5+
of the mandatory local-CI gate (see the repo's local-first workflow): it is
6+
driven entirely over ssh by reproducible scripts in this directory.
77

8-
## Windows VM (`run.sh windows-vm`) — real ACL/token/owner semantics
8+
## Windows VM — real ACL/token/owner semantics (script-driven)
99

10-
Wine only compile-checks Windows: kernel-real security semantics (object
11-
ownership, DACLs, token identity — e.g. the Administrators-default-owner
12-
policy on CI runners) need real Windows.
10+
Wine only compile-checks Windows. Kernel-real security semantics — object
11+
ownership, DACLs, token identity, the Administrators-default-owner policy on
12+
CI runners — need real Windows. The VM turns blind CI round-trips into a
13+
seconds-fast local loop with full stderr/debugger visibility.
1314

14-
One-time setup (~15 min interactive):
15+
### Files
16+
17+
| File | Runs where | Purpose |
18+
|---|---|---|
19+
| `windows-bootstrap.ps1` | inside the VM, once | OpenSSH + host key + CI owner-policy mirror |
20+
| `provision-windows.sh` | on the host | zero → fully-built, idempotent (disk-loss recovery) |
21+
| `win.sh` | on the host | daily driver: build / test / guards / install-E2E / shell |
22+
23+
Host-local config (never committed): `~/.claude/cbm-vm/config` with
24+
`CBM_VM_HOST=<ip>` and `CBM_VM_USER=<user>`; ssh keypair at
25+
`~/.claude/cbm-vm/id_ed25519` (generate: `ssh-keygen -t ed25519 -N "" -C
26+
claude-cbm-vm -f ~/.claude/cbm-vm/id_ed25519`, public half goes into
27+
`windows-bootstrap.ps1`).
28+
29+
### One-time VM creation (~30 min interactive, do it once, snapshot it)
1530

1631
1. `brew install --cask utm` (free/open source).
17-
2. Download the official **Windows 11 ARM64** ISO from
18-
https://www.microsoft.com/software-download/windows11arm64 (pick edition
19-
+ language; a plain resumable download). Windows runs unactivated for
20-
testing (cosmetic watermark); add a license if you want it clean.
21-
3. Open **UTM** → New VM → Virtualize → Windows → select the ISO →
22-
ALL host cores / 16-24 GB. vCPUs are host-scheduled, not pinned: idle
32+
2. Download the official **Windows 11 ARM64** ISO:
33+
https://www.microsoft.com/software-download/windows11arm64
34+
3. UTM → New VM → Virtualize → Windows → the ISO → **all host cores**,
35+
12-16 GB RAM, 64+ GB disk. vCPUs are host-scheduled, not pinned: idle
2336
guest cores cost nothing and nothing is seized from parallel work — no
2437
artificial limits, speed first. RAM is the one bounded number (Windows
25-
commits it eagerly). Click through Windows setup (local account, e.g. `dev`).
26-
4. Inside Windows, run `vm/windows-bootstrap.ps1` (elevated PowerShell):
27-
installs msys2 + CLANG64 toolchain + git, enables OpenSSH server, and
28-
mirrors the GitHub-runner security policy
29-
(`NoDefaultAdminOwner=0` → new objects owned by BUILTIN\Administrators —
30-
the policy that surfaces the exact-owner validation class).
31-
5. Note the VM's IP (`ipconfig`) and write it to `vm/config.env`:
32-
`CBM_WIN_VM_SSH=dev@<ip>`.
33-
34-
The leg then drives the VM over ssh: sync the worktree, build with the
35-
CLANG64 toolchain, run the requested suites — a debugger-capable local loop
36-
for the class that previously needed blind CI round-trips.
38+
commits it eagerly).
39+
4. Windows setup. At "Let's connect you to a network": Shift+Fn+F10 →
40+
`start ms-cxh:localonly` → create a local account (no Microsoft account,
41+
no network needed).
42+
5. **⭐ Shut down and CLONE the VM in UTM now** ("Windows-CLEAN-backup").
43+
Driver installs can brick the boot; the clone turns that into a 10-second
44+
rollback instead of a reinstall. Boot the original afterwards.
45+
6. Install the SPICE guest tools (network driver): download
46+
https://github.com/utmapp/qemu/releases/download/v7.0.0-utm/spice-guest-tools-0.164.4.iso
47+
on the host, mount via UTM's CD icon, run as administrator inside the VM,
48+
restart. A brief "Display output is not active" during restart is normal.
49+
7. Get `windows-bootstrap.ps1` into the VM via the ISO trick in its header
50+
(clipboard and shared folders DO NOT work on Windows-ARM guests — the
51+
SPICE vdagent/webdavd services don't exist there; don't chase them),
52+
run it, note the printed ip/user, write `~/.claude/cbm-vm/config`,
53+
reboot the VM once (owner policy).
54+
8. From the host: `test-infrastructure/vm/provision-windows.sh` — installs
55+
msys2 + both toolchains, clones the repo, builds everything, smoke-checks.
56+
Re-run it any time; it is the single recovery command if the disk is lost.
57+
58+
### Daily loop
59+
60+
```
61+
vm/win.sh update # sync VM repo to the pushed branch + rebuild
62+
vm/win.sh test cli daemon_ipc # run suites natively, seconds
63+
vm/win.sh guards # the Windows guard scripts
64+
vm/win.sh smoke-install # managed-install E2E, stderr VISIBLE
65+
vm/win.sh sh "cd /c/cbm && gdb ..." # anything, interactively
66+
vm/win.sh push-file src/cli/cli.c /c/cbm/src/cli/cli.c # WIP iteration
67+
```
68+
69+
### Toolchains & honest limits
70+
71+
- **CLANGARM64 (native, default)**: fast on all cores; OS-semantics bugs
72+
(ACL/owner/paths/locks — the entire Windows tail class) reproduce
73+
faithfully. Build with `SANITIZE=` (no ASan runtime exists for
74+
aarch64-windows).
75+
- **CLANG64 (x86_64 = CI's arch, emulated)**: for arch-parity checks
76+
(`win.sh asan-build` builds it). **ASan does NOT work under x64-on-ARM
77+
emulation** (the runtime faults in process init) — verified, don't retry.
78+
Windows sanitizer coverage therefore stays a CI-only leg; note the CI
79+
windows failures have historically been logic failures, not ASan aborts.
80+
- The bootstrap mirrors the GitHub-runner default-owner policy
81+
(`NoDefaultAdminOwner=0`) so CI-only ownership refusals reproduce locally.
82+
- Environment shape (runner temp paths, runneradmin profile) is
83+
approximated, not identical — CI on the final SHA remains the proof.
3784

3885
## macOS runner VM (`run.sh mac-vm`) — the GitHub-runner macOS environment
3986

4087
A local Mac is not the GitHub macOS runner (different temp ancestry, ACLs,
4188
policies): failures like the index-supervisor worker exits reproduce only
4289
there. Cirrus Labs publishes runner-equivalent images.
4390

44-
One-time setup (automated, big download):
45-
4691
1. `brew trust cirruslabs/cli && brew install tart` (free OSS).
4792
2. `tart clone ghcr.io/cirruslabs/macos-runner:sequoia cbm-mac-runner`
4893
(tens of GB, one time).
4994

5095
The leg boots the VM headless (`tart run --no-graphics cbm-mac-runner`),
51-
syncs the worktree over ssh (default credentials admin/admin inside the
52-
runner image), and executes `scripts/test.sh` — the same environment class
53-
GitHub schedules, locally and debuggable.
96+
syncs the worktree over ssh (admin/admin inside the runner image), and
97+
executes `scripts/test.sh`.
5498

5599
## Honesty rules
56100

57-
- These legs are OPT-IN: absent tooling prints guidance and exits with a
58-
distinct status; it is never silently skipped when explicitly requested.
101+
- These legs never silently skip: absent tooling prints setup guidance and
102+
exits with a distinct status.
59103
- VM results complement, never replace, the CI matrix on the final SHA.
104+
- Everything runs over the local UTM/host network only — nothing about the
105+
VM loop touches the internet except the VM's own package/repo downloads.
Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
#!/usr/bin/env bash
2+
# provision-windows.sh — provision the Windows test VM from the macOS host.
3+
#
4+
# Idempotent: safe to re-run at any time; every step skips work already done.
5+
# Recovers a fresh VM (or a lost disk) to fully-built state in one command.
6+
#
7+
# Prerequisites (one-time, manual — see README.md):
8+
# 1. UTM VM with Windows 11 ARM64 installed, CLONED as a backup before
9+
# drivers, SPICE guest tools installed (network), and
10+
# windows-bootstrap.ps1 run inside the VM (OpenSSH + authorized key).
11+
# 2. Host config in ~/.claude/cbm-vm/config (KEY=VALUE):
12+
# CBM_VM_HOST=192.168.64.4
13+
# CBM_VM_USER=test
14+
# Key at ~/.claude/cbm-vm/id_ed25519 (see README.md to generate).
15+
#
16+
# Usage:
17+
# test-infrastructure/vm/provision-windows.sh # full provision
18+
# test-infrastructure/vm/provision-windows.sh --update # repo+build only
19+
set -euo pipefail
20+
21+
CONFIG="${HOME}/.claude/cbm-vm/config"
22+
KEY="${HOME}/.claude/cbm-vm/id_ed25519"
23+
[ -f "$CONFIG" ] && . "$CONFIG"
24+
HOST="${CBM_VM_HOST:?set CBM_VM_HOST in ~/.claude/cbm-vm/config}"
25+
USER_="${CBM_VM_USER:-test}"
26+
BRANCH="${CBM_VM_BRANCH:-feat/shared-coordination-daemon}"
27+
REPO_URL="${CBM_VM_REPO:-https://github.com/DeusData/codebase-memory-mcp.git}"
28+
MSYS2_SFX_URL="https://repo.msys2.org/distrib/msys2-x86_64-latest.sfx.exe"
29+
30+
SSH=(ssh -i "$KEY" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
31+
-o ConnectTimeout=10 -o BatchMode=yes "${USER_}@${HOST}")
32+
33+
# Run a command inside the given msys2 environment (clangarm64|clang64|msys).
34+
vm() { # vm <env> <command...>
35+
local env="$1"; shift
36+
"${SSH[@]}" "C:\\msys64\\msys2_shell.cmd -defterm -no-start -${env} -c \"$*\""
37+
}
38+
vm_cmd() { "${SSH[@]}" "$@"; } # plain cmd.exe
39+
40+
step() { printf '\n\033[1m== %s ==\033[0m\n' "$*"; }
41+
42+
step "0/6 reachability"
43+
vm_cmd "echo VM_OK & ver" | grep -q VM_OK || {
44+
echo "FATAL: cannot reach ${USER_}@${HOST} over ssh." >&2
45+
echo "Run windows-bootstrap.ps1 inside the VM first (see README.md)." >&2
46+
exit 1
47+
}
48+
49+
if [ "${1:-}" != "--update" ]; then
50+
step "1/6 msys2 base (skip if present)"
51+
if ! vm_cmd "if exist C:\\msys64\\usr\\bin\\bash.exe echo MSYS2_PRESENT" | grep -q MSYS2_PRESENT; then
52+
vm_cmd "powershell -Command \"[Net.ServicePointManager]::SecurityProtocol='Tls12'; Invoke-WebRequest -Uri ${MSYS2_SFX_URL} -OutFile C:\\msys2.sfx.exe\""
53+
vm_cmd "C:\\msys2.sfx.exe -y -oC:\\" >/dev/null
54+
vm "msys" "pacman-key --init && pacman-key --populate msys2"
55+
fi
56+
57+
step "2/6 pacman update (repeat-safe)"
58+
vm "msys" "rm -f /var/lib/pacman/db.lck; pacman -Syu --noconfirm --noprogressbar" || true
59+
60+
step "3/6 toolchains: CLANGARM64 (native, fast) + CLANG64 (x86_64 = CI arch, ASan)"
61+
vm "msys" "pacman -S --noconfirm --noprogressbar --needed \
62+
git make coreutils \
63+
mingw-w64-clang-aarch64-clang mingw-w64-clang-aarch64-zlib \
64+
mingw-w64-clang-aarch64-python mingw-w64-clang-aarch64-ccache \
65+
mingw-w64-clang-x86_64-clang mingw-w64-clang-x86_64-compiler-rt \
66+
mingw-w64-clang-x86_64-zlib mingw-w64-clang-x86_64-ccache"
67+
fi
68+
69+
step "4/6 repo clone/update -> /c/cbm @ ${BRANCH}"
70+
if vm "clangarm64" "test -d /c/cbm/.git && echo REPO_PRESENT" | grep -q REPO_PRESENT; then
71+
vm "clangarm64" "cd /c/cbm && git fetch origin ${BRANCH} && git reset --hard FETCH_HEAD && git log --oneline -1"
72+
else
73+
vm "clangarm64" "git clone --branch ${BRANCH} --single-branch --depth 200 ${REPO_URL} /c/cbm && cd /c/cbm && git log --oneline -1"
74+
fi
75+
76+
step "5/6 build: native ARM64 binary + launcher + test-runner (no ASan on arm64)"
77+
vm "clangarm64" "cd /c/cbm && make -j\\\$(nproc) -f Makefile.cbm CC=clang CXX=clang++ SANITIZE= cbm build/c/test-runner > /tmp/provision-build.log 2>&1 && echo BUILD_OK || (echo BUILD_FAIL; tail -15 /tmp/provision-build.log; exit 1)"
78+
79+
step "6/6 smoke: binary + test-runner start"
80+
vm "clangarm64" "cd /c/cbm && ./build/c/codebase-memory-mcp.exe --version && ./build/c/test-runner --list-suites | head -3"
81+
82+
printf '\n\033[1mPROVISION COMPLETE\033[0m — daily driving: test-infrastructure/vm/win.sh\n'

‎test-infrastructure/vm/win.sh‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
#!/usr/bin/env bash
2+
# win.sh — daily driver for the Windows test VM (real-Windows leg of local CI).
3+
#
4+
# All commands run over ssh (local UTM network only — nothing leaves the
5+
# machine). Config: ~/.claude/cbm-vm/config (CBM_VM_HOST, CBM_VM_USER),
6+
# key: ~/.claude/cbm-vm/id_ed25519. Provision first: provision-windows.sh.
7+
#
8+
# Usage:
9+
# win.sh status # reachability + repo + build state
10+
# win.sh update # fetch+reset repo to pushed branch, rebuild
11+
# win.sh build # incremental native build (binary+runner)
12+
# win.sh test <suite...> # run test-runner suites (native ARM64)
13+
# win.sh guards # run the Windows guard scripts (python)
14+
# win.sh smoke-install # real managed-install E2E (Phase 8 class)
15+
# win.sh sh <command...> # arbitrary command in CLANGARM64 env
16+
# win.sh push-file <local> <vm> # scp one file into the VM (WIP iteration)
17+
# win.sh asan-build # x86_64 CLANG64 build WITH ASan (CI arch)
18+
set -euo pipefail
19+
20+
CONFIG="${HOME}/.claude/cbm-vm/config"
21+
KEY="${HOME}/.claude/cbm-vm/id_ed25519"
22+
[ -f "$CONFIG" ] && . "$CONFIG"
23+
HOST="${CBM_VM_HOST:?set CBM_VM_HOST in ~/.claude/cbm-vm/config}"
24+
USER_="${CBM_VM_USER:-test}"
25+
BRANCH="${CBM_VM_BRANCH:-feat/shared-coordination-daemon}"
26+
JOBS='\$(nproc)'
27+
28+
SSH=(ssh -i "$KEY" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
29+
-o ConnectTimeout=10 -o BatchMode=yes "${USER_}@${HOST}")
30+
31+
vm() { local env="$1"; shift
32+
"${SSH[@]}" "C:\\msys64\\msys2_shell.cmd -defterm -no-start -${env} -c \"$*\""; }
33+
34+
cmd="${1:-status}"; shift || true
35+
case "$cmd" in
36+
status)
37+
"${SSH[@]}" "echo VM_REACHABLE & ver"
38+
vm clangarm64 "cd /c/cbm 2>/dev/null && git log --oneline -1 && ls -la build/c/codebase-memory-mcp.exe build/c/test-runner.exe 2>/dev/null || echo 'repo/build missing — run provision-windows.sh'"
39+
;;
40+
update)
41+
vm clangarm64 "cd /c/cbm && git fetch origin ${BRANCH} && git reset --hard FETCH_HEAD && git log --oneline -1"
42+
exec "$0" build
43+
;;
44+
build)
45+
vm clangarm64 "cd /c/cbm && make -j${JOBS} -f Makefile.cbm CC=clang CXX=clang++ SANITIZE= cbm build/c/test-runner > /tmp/win-build.log 2>&1 && echo BUILD_OK || (echo BUILD_FAIL; tail -20 /tmp/win-build.log; exit 1)"
46+
;;
47+
test)
48+
[ $# -ge 1 ] || { echo "usage: win.sh test <suite...>" >&2; exit 2; }
49+
vm clangarm64 "cd /c/cbm && ./build/c/test-runner $* 2>&1 | tail -40"
50+
;;
51+
guards)
52+
vm clangarm64 "cd /c/cbm && for g in tests/windows/test_*.py; do echo \"== \\\$g ==\"; python \\\$g build/c/codebase-memory-mcp.exe 2>&1 | tail -6; done"
53+
;;
54+
smoke-install)
55+
# Real managed install E2E with FULL stderr visible — the exact class the
56+
# CI smoke Phase 8 exercises but cannot show (probe hides launcher stderr).
57+
vm clangarm64 "cd /c/cbm && ./build/c/codebase-memory-mcp.exe install 2>&1 | tail -25"
58+
;;
59+
sh)
60+
vm clangarm64 "$*"
61+
;;
62+
push-file)
63+
[ $# -eq 2 ] || { echo "usage: win.sh push-file <local-path> <vm-path>" >&2; exit 2; }
64+
scp -i "$KEY" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null "$1" "${USER_}@${HOST}:$2"
65+
;;
66+
asan-build)
67+
# x86_64 (CI's exact arch) with ASan, runs under Windows-on-ARM emulation.
68+
# Slower; use when a finding needs sanitizer confirmation or CI-arch parity.
69+
vm clang64 "cd /c/cbm && make -j${JOBS} -f Makefile.cbm CC=clang CXX=clang++ build/c/test-runner > /tmp/win-asan-build.log 2>&1 && echo ASAN_BUILD_OK || (echo ASAN_BUILD_FAIL; tail -20 /tmp/win-asan-build.log; exit 1)"
70+
;;
71+
*)
72+
echo "unknown command: $cmd (see header for usage)" >&2; exit 2
73+
;;
74+
esac

0 commit comments

Comments
 (0)