|
1 | | -# VM test legs (opt-in) |
| 1 | +# VM test legs — real-Windows and runner-macOS |
2 | 2 |
|
3 | 3 | Two failure classes cannot be reproduced by the container legs and need real |
4 | | -virtual machines. Both stacks are free; the legs activate automatically in |
5 | | -`run.sh` when the tooling and the named VM are detected, and print setup |
6 | | -guidance otherwise. |
| 4 | +virtual machines. Both stacks are free. The Windows VM is a first-class part |
| 5 | +of the mandatory local-CI gate (see the repo's local-first workflow): it is |
| 6 | +driven entirely over ssh by reproducible scripts in this directory. |
7 | 7 |
|
8 | | -## Windows VM (`run.sh windows-vm`) — real ACL/token/owner semantics |
| 8 | +## Windows VM — real ACL/token/owner semantics (script-driven) |
9 | 9 |
|
10 | | -Wine only compile-checks Windows: kernel-real security semantics (object |
11 | | -ownership, DACLs, token identity — e.g. the Administrators-default-owner |
12 | | -policy on CI runners) need real Windows. |
| 10 | +Wine only compile-checks Windows. Kernel-real security semantics — object |
| 11 | +ownership, DACLs, token identity, the Administrators-default-owner policy on |
| 12 | +CI runners — need real Windows. The VM turns blind CI round-trips into a |
| 13 | +seconds-fast local loop with full stderr/debugger visibility. |
13 | 14 |
|
14 | | -One-time setup (~15 min interactive): |
| 15 | +### Files |
| 16 | + |
| 17 | +| File | Runs where | Purpose | |
| 18 | +|---|---|---| |
| 19 | +| `windows-bootstrap.ps1` | inside the VM, once | OpenSSH + host key + CI owner-policy mirror | |
| 20 | +| `provision-windows.sh` | on the host | zero → fully-built, idempotent (disk-loss recovery) | |
| 21 | +| `win.sh` | on the host | daily driver: build / test / guards / install-E2E / shell | |
| 22 | + |
| 23 | +Host-local config (never committed): `~/.claude/cbm-vm/config` with |
| 24 | +`CBM_VM_HOST=<ip>` and `CBM_VM_USER=<user>`; ssh keypair at |
| 25 | +`~/.claude/cbm-vm/id_ed25519` (generate: `ssh-keygen -t ed25519 -N "" -C |
| 26 | +claude-cbm-vm -f ~/.claude/cbm-vm/id_ed25519`, public half goes into |
| 27 | +`windows-bootstrap.ps1`). |
| 28 | + |
| 29 | +### One-time VM creation (~30 min interactive, do it once, snapshot it) |
15 | 30 |
|
16 | 31 | 1. `brew install --cask utm` (free/open source). |
17 | | -2. Download the official **Windows 11 ARM64** ISO from |
18 | | - https://www.microsoft.com/software-download/windows11arm64 (pick edition |
19 | | - + language; a plain resumable download). Windows runs unactivated for |
20 | | - testing (cosmetic watermark); add a license if you want it clean. |
21 | | -3. Open **UTM** → New VM → Virtualize → Windows → select the ISO → |
22 | | - ALL host cores / 16-24 GB. vCPUs are host-scheduled, not pinned: idle |
| 32 | +2. Download the official **Windows 11 ARM64** ISO: |
| 33 | + https://www.microsoft.com/software-download/windows11arm64 |
| 34 | +3. UTM → New VM → Virtualize → Windows → the ISO → **all host cores**, |
| 35 | + 12-16 GB RAM, 64+ GB disk. vCPUs are host-scheduled, not pinned: idle |
23 | 36 | guest cores cost nothing and nothing is seized from parallel work — no |
24 | 37 | artificial limits, speed first. RAM is the one bounded number (Windows |
25 | | - commits it eagerly). Click through Windows setup (local account, e.g. `dev`). |
26 | | -4. Inside Windows, run `vm/windows-bootstrap.ps1` (elevated PowerShell): |
27 | | - installs msys2 + CLANG64 toolchain + git, enables OpenSSH server, and |
28 | | - mirrors the GitHub-runner security policy |
29 | | - (`NoDefaultAdminOwner=0` → new objects owned by BUILTIN\Administrators — |
30 | | - the policy that surfaces the exact-owner validation class). |
31 | | -5. Note the VM's IP (`ipconfig`) and write it to `vm/config.env`: |
32 | | - `CBM_WIN_VM_SSH=dev@<ip>`. |
33 | | - |
34 | | -The leg then drives the VM over ssh: sync the worktree, build with the |
35 | | -CLANG64 toolchain, run the requested suites — a debugger-capable local loop |
36 | | -for the class that previously needed blind CI round-trips. |
| 38 | + commits it eagerly). |
| 39 | +4. Windows setup. At "Let's connect you to a network": Shift+Fn+F10 → |
| 40 | + `start ms-cxh:localonly` → create a local account (no Microsoft account, |
| 41 | + no network needed). |
| 42 | +5. **⭐ Shut down and CLONE the VM in UTM now** ("Windows-CLEAN-backup"). |
| 43 | + Driver installs can brick the boot; the clone turns that into a 10-second |
| 44 | + rollback instead of a reinstall. Boot the original afterwards. |
| 45 | +6. Install the SPICE guest tools (network driver): download |
| 46 | + https://github.com/utmapp/qemu/releases/download/v7.0.0-utm/spice-guest-tools-0.164.4.iso |
| 47 | + on the host, mount via UTM's CD icon, run as administrator inside the VM, |
| 48 | + restart. A brief "Display output is not active" during restart is normal. |
| 49 | +7. Get `windows-bootstrap.ps1` into the VM via the ISO trick in its header |
| 50 | + (clipboard and shared folders DO NOT work on Windows-ARM guests — the |
| 51 | + SPICE vdagent/webdavd services don't exist there; don't chase them), |
| 52 | + run it, note the printed ip/user, write `~/.claude/cbm-vm/config`, |
| 53 | + reboot the VM once (owner policy). |
| 54 | +8. From the host: `test-infrastructure/vm/provision-windows.sh` — installs |
| 55 | + msys2 + both toolchains, clones the repo, builds everything, smoke-checks. |
| 56 | + Re-run it any time; it is the single recovery command if the disk is lost. |
| 57 | + |
| 58 | +### Daily loop |
| 59 | + |
| 60 | +``` |
| 61 | +vm/win.sh update # sync VM repo to the pushed branch + rebuild |
| 62 | +vm/win.sh test cli daemon_ipc # run suites natively, seconds |
| 63 | +vm/win.sh guards # the Windows guard scripts |
| 64 | +vm/win.sh smoke-install # managed-install E2E, stderr VISIBLE |
| 65 | +vm/win.sh sh "cd /c/cbm && gdb ..." # anything, interactively |
| 66 | +vm/win.sh push-file src/cli/cli.c /c/cbm/src/cli/cli.c # WIP iteration |
| 67 | +``` |
| 68 | + |
| 69 | +### Toolchains & honest limits |
| 70 | + |
| 71 | +- **CLANGARM64 (native, default)**: fast on all cores; OS-semantics bugs |
| 72 | + (ACL/owner/paths/locks — the entire Windows tail class) reproduce |
| 73 | + faithfully. Build with `SANITIZE=` (no ASan runtime exists for |
| 74 | + aarch64-windows). |
| 75 | +- **CLANG64 (x86_64 = CI's arch, emulated)**: for arch-parity checks |
| 76 | + (`win.sh asan-build` builds it). **ASan does NOT work under x64-on-ARM |
| 77 | + emulation** (the runtime faults in process init) — verified, don't retry. |
| 78 | + Windows sanitizer coverage therefore stays a CI-only leg; note the CI |
| 79 | + windows failures have historically been logic failures, not ASan aborts. |
| 80 | +- The bootstrap mirrors the GitHub-runner default-owner policy |
| 81 | + (`NoDefaultAdminOwner=0`) so CI-only ownership refusals reproduce locally. |
| 82 | +- Environment shape (runner temp paths, runneradmin profile) is |
| 83 | + approximated, not identical — CI on the final SHA remains the proof. |
37 | 84 |
|
38 | 85 | ## macOS runner VM (`run.sh mac-vm`) — the GitHub-runner macOS environment |
39 | 86 |
|
40 | 87 | A local Mac is not the GitHub macOS runner (different temp ancestry, ACLs, |
41 | 88 | policies): failures like the index-supervisor worker exits reproduce only |
42 | 89 | there. Cirrus Labs publishes runner-equivalent images. |
43 | 90 |
|
44 | | -One-time setup (automated, big download): |
45 | | - |
46 | 91 | 1. `brew trust cirruslabs/cli && brew install tart` (free OSS). |
47 | 92 | 2. `tart clone ghcr.io/cirruslabs/macos-runner:sequoia cbm-mac-runner` |
48 | 93 | (tens of GB, one time). |
49 | 94 |
|
50 | 95 | The leg boots the VM headless (`tart run --no-graphics cbm-mac-runner`), |
51 | | -syncs the worktree over ssh (default credentials admin/admin inside the |
52 | | -runner image), and executes `scripts/test.sh` — the same environment class |
53 | | -GitHub schedules, locally and debuggable. |
| 96 | +syncs the worktree over ssh (admin/admin inside the runner image), and |
| 97 | +executes `scripts/test.sh`. |
54 | 98 |
|
55 | 99 | ## Honesty rules |
56 | 100 |
|
57 | | -- These legs are OPT-IN: absent tooling prints guidance and exits with a |
58 | | - distinct status; it is never silently skipped when explicitly requested. |
| 101 | +- These legs never silently skip: absent tooling prints setup guidance and |
| 102 | + exits with a distinct status. |
59 | 103 | - VM results complement, never replace, the CI matrix on the final SHA. |
| 104 | +- Everything runs over the local UTM/host network only — nothing about the |
| 105 | + VM loop touches the internet except the VM's own package/repo downloads. |
0 commit comments