Skip to content

Commit 0eb2c58

Browse files
Flipper1994claude
andcommitted
test(windows): convert fixed reds to green guards, wire CI, rewrite drive test
Rebased onto current main and reworked in response to review. Three of the four Windows reds were fixed upstream since the branch was cut at b075f05, so they are now green regression guards; the fourth stays a genuine known-red. - test_non_ascii_path.py (DeusData#636/DeusData#357): green guard - fixed by DeusData#700 (per-pass readers now route through cbm_fopen -> _wfopen). Re-verified green on main. - test_hook_augment.py (DeusData#618): green guard - fixed by DeusData#619 (cbm_is_walkable_abs_path accepts drive-letter X:/ cwd). Re-verified green on main. - test_ui_drive_listing.py (DeusData#548): rewritten. The fix exposes drives via a new roots[] field, not the dirs[] array the old test asserted (which would stay red against fixed code). Now asserts every fixed drive is in roots and browsable. Re-verified green on main (drives C:/D:/E:). - test_cli_non_ascii_arg.py (DeusData#423/#20): unchanged - main() is still narrow-argv with no wide command line, so this remains genuinely red (the keeper). - scripts/test-windows.ps1: split green guards (gate CI) from opt-in known-reds; add -GuardsOnly; run indexing in-process (CBM_INDEX_SUPERVISOR=0) so a guard reflects the path/hook/drive fix under test, not the index-worker spawn path. - .github/workflows/_test.yml: new test-windows-guards job builds the product+UI binary (scripts/build.sh --with-ui) and runs the guards with -GuardsOnly so DeusData#700/DeusData#619/DeusData#548 stay enforced on Windows CI. - RED_TEST_ANALYSIS.md: refreshed to record the landed fixes and current status. Signed-off-by: Flipper <jacobphilipp@ymail.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 2b4fa2d commit 0eb2c58

6 files changed

Lines changed: 337 additions & 178 deletions

File tree

‎.github/workflows/_test.yml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,3 +124,43 @@ jobs:
124124
run: scripts/test.sh CC=clang CXX=clang++ ${{ matrix.os == 'windows-11-arm' && 'SANITIZE=' || '' }}
125125
env:
126126
CBM_SKIP_PERF: ${{ inputs.skip_perf && '1' || '' }}
127+
128+
# Windows product-surface regression guards. Distinct from test-windows above
129+
# (the sanitizer C suite): these drive a real product binary + embedded HTTP UI
130+
# over stdio / CLI / HTTP and fail if a Windows bug already fixed on main comes
131+
# back -- non-ASCII repo paths dropping definitions (#636/#357, fixed by #700),
132+
# the PreToolUse hook augmenter no-op on drive-letter cwd (#618, fixed by #619),
133+
# and the UI directory picker not enumerating drives (#548, roots field). The
134+
# still-open narrow-argv repro (#423/#20) is opt-in and excluded via -GuardsOnly.
135+
test-windows-guards:
136+
runs-on: windows-latest
137+
timeout-minutes: 60
138+
steps:
139+
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
140+
141+
- uses: msys2/setup-msys2@66cd2cce69caa17b53920067426061ca1de3a884 # v2
142+
with:
143+
msystem: CLANG64
144+
path-type: inherit
145+
install: >-
146+
mingw-w64-clang-x86_64-clang
147+
mingw-w64-clang-x86_64-zlib
148+
make
149+
git
150+
151+
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
152+
with:
153+
node-version: "22"
154+
155+
- name: Build product binary with embedded UI
156+
shell: msys2 {0}
157+
# --with-ui builds the frontend (npm) and embeds it, so the drive-picker
158+
# guard's HTTP UI is available. Functional gate only (no sanitizers).
159+
run: scripts/build.sh --with-ui CC=clang CXX=clang++
160+
161+
- name: Windows regression guards (#636/#357, #618, #548)
162+
shell: pwsh
163+
# -GuardsOnly runs the three green guards and gates on them; the runner
164+
# runs indexing in-process (CBM_INDEX_SUPERVISOR=0) so a guard reflects the
165+
# path/hook/drive fix under test, not the orthogonal index-worker spawn.
166+
run: ./scripts/test-windows.ps1 -GuardsOnly -Binary build/c/codebase-memory-mcp.exe

‎scripts/test-windows.ps1‎

Lines changed: 96 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,33 +1,66 @@
11
<#
22
.SYNOPSIS
3-
Run the native-Windows red-test suite for codebase-memory-mcp.
3+
Run the native-Windows product-surface test suite for codebase-memory-mcp.
44
55
.DESCRIPTION
6-
Builds the production binary (build/c/codebase-memory-mcp.exe) if it is not
7-
already present, then runs the deterministic Windows red tests under
8-
tests/windows/. These tests reproduce platform-specific failures at the
9-
product surface (real MCP process, real stdio, real SQLite DB).
6+
Builds the product binary (build/c/codebase-memory-mcp.exe) if it is not
7+
already present, then runs the deterministic Windows integration tests under
8+
tests/windows/ against a real codebase-memory-mcp.exe (real stdio / CLI /
9+
HTTP UI, real SQLite DB).
1010
11-
The unit/invariant C suite is built and run via Makefile.cbm. On native
12-
Windows the MinGW/LLVM toolchain ships no libasan/libubsan, so the sanitizer
13-
flags must be disabled for the local build (SANITIZE=). Where the toolchain
14-
*does* provide AddressSanitizer/UBSan (Linux containers, WSL), prefer
15-
scripts/test.sh which keeps the sanitizers on.
11+
Two categories of test:
12+
13+
GUARDS - regression guards for Windows bugs already fixed on main.
14+
They must stay GREEN (exit 0); a RED (exit 1) means the fix
15+
regressed and fails this runner.
16+
* test_non_ascii_path.py guards #636/#357 (fixed by #700)
17+
* test_hook_augment.py guards #618 (fixed by #619)
18+
* test_ui_drive_listing.py guards #548 (roots field)
19+
20+
KNOWN REDS - genuine, still-open Windows bugs reproduced at the product
21+
surface. They are EXPECTED to be RED (exit 1) and are opt-in
22+
(never gate CI). If one turns GREEN the underlying bug was
23+
fixed and it should be promoted to a guard.
24+
* test_cli_non_ascii_arg.py reproduces #423/#20 (narrow
25+
argv main() - no wide command line)
26+
27+
Determinism: indexing runs in-process (CBM_INDEX_SUPERVISOR=0). These tests
28+
exercise path / hook / drive handling, not the index-supervisor subprocess
29+
path; the pass-level readers (#700's cbm_fopen routing) run in-process either
30+
way, so the guard coverage is identical while results stay independent of the
31+
local toolchain's worker-spawn behavior.
32+
33+
On native Windows the MinGW/LLVM toolchain ships no libasan/libubsan, so the
34+
build disables sanitizers (SANITIZE=). Where the toolchain provides
35+
AddressSanitizer/UBSan (Linux containers, WSL), prefer scripts/test.sh.
1636
1737
.PARAMETER Binary
18-
Path to an existing codebase-memory-mcp.exe. If omitted, the script looks for
19-
build/c/codebase-memory-mcp.exe and builds it when missing.
38+
Path to an existing codebase-memory-mcp.exe. If omitted, the script builds it
39+
(target selected by -Target) into build/c/.
40+
41+
.PARAMETER Target
42+
Makefile.cbm target used when building: 'cbm-with-ui' (default; needed for the
43+
drive-picker guard's embedded HTTP UI) or 'cbm' (no UI - the drive guard then
44+
reports a precondition and is skipped).
45+
46+
.PARAMETER GuardsOnly
47+
Run only the green guards (the CI gate). Skips the opt-in known-red repros.
2048
2149
.PARAMETER Make
2250
Path to GNU make (default: 'make' on PATH; MSYS2 ships it at
2351
C:\msys64\usr\bin\make.exe).
2452
2553
.EXAMPLE
2654
pwsh -File scripts/test-windows.ps1
55+
.EXAMPLE
56+
pwsh -File scripts/test-windows.ps1 -GuardsOnly -Binary build\c\codebase-memory-mcp.exe
2757
#>
2858
[CmdletBinding()]
2959
param(
3060
[string]$Binary,
61+
[ValidateSet("cbm-with-ui", "cbm")]
62+
[string]$Target = "cbm-with-ui",
63+
[switch]$GuardsOnly,
3164
[string]$Make = "make"
3265
)
3366

@@ -37,7 +70,7 @@ Set-Location $repoRoot
3770

3871
$python = (Get-Command python -ErrorAction SilentlyContinue)
3972
if (-not $python) { $python = (Get-Command py -ErrorAction SilentlyContinue) }
40-
if (-not $python) { throw "Python 3 is required to run the Windows red tests." }
73+
if (-not $python) { throw "Python 3 is required to run the Windows tests." }
4174
$py = $python.Source
4275

4376
# A writable Windows temp dir that GNU make forwards to the native gcc. MSYS2
@@ -51,8 +84,8 @@ function Resolve-Binary {
5184
if ($Explicit) { return (Resolve-Path $Explicit).Path }
5285
$built = Join-Path $repoRoot "build\c\codebase-memory-mcp.exe"
5386
if (Test-Path $built) { return $built }
54-
Write-Host "Building production binary via Makefile.cbm ..." -ForegroundColor Cyan
55-
& $Make "-j" "-f" "Makefile.cbm" "cbm" "TMP=$tmp" "TEMP=$tmp" "TMPDIR=$tmp"
87+
Write-Host "Building $Target via Makefile.cbm ..." -ForegroundColor Cyan
88+
& $Make "-j" "-f" "Makefile.cbm" $Target "SANITIZE=" "TMP=$tmp" "TEMP=$tmp" "TMPDIR=$tmp"
5689
if ($LASTEXITCODE -ne 0) { throw "build failed (exit $LASTEXITCODE)" }
5790
if (-not (Test-Path $built)) { throw "binary not produced at $built" }
5891
return $built
@@ -61,45 +94,73 @@ function Resolve-Binary {
6194
$bin = Resolve-Binary -Explicit $Binary
6295
Write-Host "Binary: $bin" -ForegroundColor Green
6396

64-
$env:PYTHONUTF8 = "1" # ensure the harness encodes argv/stdio as UTF-8
97+
$env:PYTHONUTF8 = "1" # encode argv/stdio as UTF-8
98+
$env:CBM_INDEX_SUPERVISOR = "0" # in-process indexing (see .DESCRIPTION)
6599

66-
# test_ui_drive_listing.py reproduces the UI directory-picker bug (#548) and
67-
# therefore needs a UI build (make -f Makefile.cbm cbm-with-ui) plus a machine
68-
# with more than one drive. Against a non-UI binary it reports a precondition
69-
# (exit 2), which is treated as a skip-with-reason, not a failure.
70-
$tests = @(
100+
# Green regression guards - must stay GREEN (exit 0). RED (exit 1) = the fix for
101+
# the referenced issue regressed. The drive-picker guard needs the embedded HTTP
102+
# UI (build target cbm-with-ui); against a non-UI binary it reports a precondition
103+
# (exit 2) and is skipped rather than failed.
104+
$guards = @(
71105
"tests\windows\test_non_ascii_path.py",
72-
"tests\windows\test_cli_non_ascii_arg.py",
73106
"tests\windows\test_hook_augment.py",
74107
"tests\windows\test_ui_drive_listing.py"
75108
)
76109

77-
$reds = @()
78-
$precond = @()
79-
foreach ($t in $tests) {
110+
# Opt-in known-red repros - EXPECTED red (exit 1); never gate CI.
111+
$knownReds = @(
112+
"tests\windows\test_cli_non_ascii_arg.py"
113+
)
114+
115+
$guardFailures = @()
116+
$guardSkips = @()
117+
$fixedKeepers = @()
118+
119+
Write-Host "`n--- Green guards ---" -ForegroundColor Cyan
120+
foreach ($t in $guards) {
80121
Write-Host "`n=== $t ===" -ForegroundColor Cyan
81122
& $py $t $bin
82123
$code = $LASTEXITCODE
83124
if ($code -eq 0) {
84125
Write-Host "GREEN ($t)" -ForegroundColor Green
85126
} elseif ($code -eq 1) {
86-
Write-Host "RED ($t) - Windows-specific failure reproduced" -ForegroundColor Red
87-
$reds += $t
127+
Write-Host "RED ($t) - REGRESSION: a fixed Windows bug is broken again" -ForegroundColor Red
128+
$guardFailures += $t
88129
} else {
89130
Write-Host "PRECONDITION ($t) exit=$code - skipped (see message above)" -ForegroundColor Yellow
90-
$precond += $t
131+
$guardSkips += $t
132+
}
133+
}
134+
135+
if (-not $GuardsOnly) {
136+
Write-Host "`n--- Known reds (opt-in, expected red) ---" -ForegroundColor Cyan
137+
foreach ($t in $knownReds) {
138+
Write-Host "`n=== $t ===" -ForegroundColor Cyan
139+
& $py $t $bin
140+
$code = $LASTEXITCODE
141+
if ($code -eq 1) {
142+
Write-Host "RED ($t) - expected; the underlying Windows bug is still open" -ForegroundColor DarkYellow
143+
} elseif ($code -eq 0) {
144+
Write-Host "GREEN ($t) - the bug appears FIXED; promote this to a guard" -ForegroundColor Green
145+
$fixedKeepers += $t
146+
} else {
147+
Write-Host "PRECONDITION ($t) exit=$code - skipped (see message above)" -ForegroundColor Yellow
148+
}
91149
}
92150
}
93151

94152
Write-Host ""
95-
if ($precond.Count -gt 0) {
96-
Write-Host ("Precondition-skipped: {0} (e.g. test_ui_drive_listing needs a UI " -f $precond.Count) -ForegroundColor Yellow
97-
Write-Host "build: make -f Makefile.cbm cbm-with-ui, and >1 drive)." -ForegroundColor Yellow
153+
if ($guardSkips.Count -gt 0) {
154+
Write-Host ("Guards skipped (precondition): {0} - e.g. the drive-picker guard " -f $guardSkips.Count) -ForegroundColor Yellow
155+
Write-Host "needs a UI build (-Target cbm-with-ui, the default)." -ForegroundColor Yellow
156+
}
157+
if ($fixedKeepers.Count -gt 0) {
158+
Write-Host ("Known-red repros that are now GREEN (promote to guards): {0}" -f ($fixedKeepers -join ", ")) -ForegroundColor Green
98159
}
99-
if ($reds.Count -gt 0) {
100-
Write-Host ("RED suite: {0} Windows red tests reproduced platform failures " -f $reds.Count) -ForegroundColor Red
101-
Write-Host "(expected until fixed). See tests/windows/RED_TEST_ANALYSIS.md." -ForegroundColor Red
160+
if ($guardFailures.Count -gt 0) {
161+
Write-Host ("REGRESSION: {0} green guard(s) went red: {1}" -f $guardFailures.Count, ($guardFailures -join ", ")) -ForegroundColor Red
162+
Write-Host "A previously-fixed Windows bug is broken again. See tests/windows/RED_TEST_ANALYSIS.md." -ForegroundColor Red
102163
exit 1
103164
}
104-
Write-Host "All runnable Windows red tests are GREEN." -ForegroundColor Green
165+
Write-Host "All Windows green guards passed." -ForegroundColor Green
105166
exit 0

0 commit comments

Comments
 (0)