Skip to content

Commit 4011a85

Browse files
authored
Merge pull request #11 from LockInTime/t3code/pr-ci
ci: gate pull requests with the existing test scripts
2 parents de57036 + 1fa587e commit 4011a85

4 files changed

Lines changed: 178 additions & 14 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
name: CI
2+
3+
# Backlog §D1: gate every pull request. Jobs reuse the same scripts a
4+
# developer runs locally, exactly like release.yml does, so CI never becomes a
5+
# separate build path that can drift from the documented workflow.
6+
7+
on:
8+
pull_request:
9+
push:
10+
branches: [main]
11+
schedule:
12+
# Nightly slot for the macOS E2E suite, which needs a GUI session and
13+
# mutates com.headless.app user defaults.
14+
- cron: "0 6 * * *"
15+
workflow_dispatch:
16+
17+
permissions:
18+
contents: read
19+
20+
concurrency:
21+
group: ci-${{ github.event.pull_request.number || github.ref }}
22+
cancel-in-progress: true
23+
24+
jobs:
25+
static:
26+
name: Static checks
27+
runs-on: ubuntu-latest
28+
timeout-minutes: 10
29+
steps:
30+
- uses: actions/checkout@v4
31+
with:
32+
fetch-depth: 0
33+
- name: Install zsh
34+
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends zsh
35+
- name: Shell syntax
36+
run: |
37+
set -eu
38+
status=0
39+
for script in $(git ls-files '*.sh'); do
40+
case "$(head -n 1 "$script")" in
41+
*zsh*) checker="zsh -n" ;;
42+
*bash*) checker="bash -n" ;;
43+
*) checker="sh -n" ;;
44+
esac
45+
if ! $checker "$script"; then
46+
echo "syntax error: $script" >&2
47+
status=1
48+
fi
49+
done
50+
exit "$status"
51+
- name: QA evidence checksums
52+
# docs/qa/evidence is the only binary media allowed in git; prove the
53+
# committed bundle still matches its manifest.
54+
run: cd docs/qa/evidence && sha256sum -c SHA256SUMS
55+
- name: Whitespace
56+
if: github.event_name == 'pull_request'
57+
run: git diff --check "origin/${{ github.base_ref }}...HEAD"
58+
59+
runtime:
60+
name: Agent runtime suite
61+
runs-on: ubuntu-latest
62+
timeout-minutes: 10
63+
steps:
64+
- uses: actions/checkout@v4
65+
- uses: pnpm/action-setup@v4
66+
- uses: actions/setup-node@v4
67+
with:
68+
node-version: 20
69+
cache: pnpm
70+
- run: pnpm install --frozen-lockfile --filter @headless/app
71+
- name: Context pruning suite
72+
run: pnpm test:runtime
73+
74+
protocol:
75+
name: Protocol suite
76+
runs-on: ubuntu-latest
77+
timeout-minutes: 30
78+
container: swift:6.1-bookworm
79+
steps:
80+
- uses: actions/checkout@v4
81+
- name: Prepare SwiftPM exclude paths
82+
# Package.swift excludes build/ and Headless.app from the host target;
83+
# Dockerfile.linux creates them for the same reason.
84+
run: mkdir -p apps/headless/build apps/headless/Headless.app
85+
- name: Protocol and security tests
86+
run: ./apps/headless/test.sh
87+
88+
web:
89+
name: Web app
90+
runs-on: ubuntu-latest
91+
timeout-minutes: 20
92+
steps:
93+
- uses: actions/checkout@v4
94+
- uses: pnpm/action-setup@v4
95+
- uses: actions/setup-node@v4
96+
with:
97+
node-version: 20
98+
cache: pnpm
99+
- run: pnpm install --frozen-lockfile --filter @headless/web
100+
- run: pnpm --filter @headless/web lint
101+
- run: pnpm --filter @headless/web build
102+
103+
linux-e2e:
104+
name: Linux E2E (Docker)
105+
runs-on: ubuntu-latest
106+
timeout-minutes: 45
107+
steps:
108+
- uses: actions/checkout@v4
109+
- name: Sandboxed Chromium E2E
110+
run: ./apps/headless/Tests/linux-docker.sh
111+
- name: Upload QA evidence
112+
if: always()
113+
uses: actions/upload-artifact@v4
114+
with:
115+
name: linux-e2e-evidence
116+
path: apps/headless/build/qa-evidence
117+
if-no-files-found: ignore
118+
retention-days: 7
119+
120+
macos:
121+
name: macOS build and protocol suite
122+
runs-on: macos-latest
123+
timeout-minutes: 45
124+
steps:
125+
- uses: actions/checkout@v4
126+
- name: Build app
127+
run: ./apps/headless/build.sh
128+
- name: Protocol and security tests
129+
run: ./apps/headless/test.sh
130+
131+
macos-e2e:
132+
name: macOS E2E (WKWebView)
133+
# Opens real windows and mutates com.headless.app user defaults, so it runs
134+
# nightly, on demand, or on a pull request labelled macos-e2e.
135+
if: >-
136+
github.event_name == 'schedule' ||
137+
github.event_name == 'workflow_dispatch' ||
138+
contains(github.event.pull_request.labels.*.name, 'macos-e2e')
139+
runs-on: macos-latest
140+
timeout-minutes: 60
141+
steps:
142+
- uses: actions/checkout@v4
143+
- name: Install FFmpeg
144+
run: brew install ffmpeg
145+
- name: Build app
146+
run: ./apps/headless/build.sh
147+
- name: E2E
148+
run: zsh ./apps/headless/Tests/macos-e2e.sh

‎AGENTS.md‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -53,8 +53,14 @@ pnpm build:linux # Linux binaries via Docker (no local Swift needed)
5353

5454
Minimum bar for a PR touching `apps/headless`: `pnpm test` and
5555
`pnpm test:runtime` pass; run the Linux E2E if you changed host, transport,
56-
runtime JS, or artifact code. There is currently no PR CI (backlog §D1) — do
57-
not rely on the machine to catch you.
56+
runtime JS, or artifact code.
57+
58+
`.github/workflows/ci.yml` gates every PR with the same scripts: static checks
59+
(shell syntax, QA evidence checksums), the agent-runtime suite, the protocol
60+
suite on Linux (Swift container) and macOS, the web lint/build, and the Linux
61+
Docker E2E. The macOS WKWebView E2E is heavier — it runs nightly, on
62+
`workflow_dispatch`, or when a PR carries the `macos-e2e` label. Run it that
63+
way before merging changes to `main.swift`, `Host/`, or capture code.
5864

5965
## Hard rules (host-enforced contracts — never weaken)
6066

‎docs/ROADMAP.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -103,8 +103,9 @@ Honest snapshot, so newcomers know what is real:
103103
command reference, and a Docker sandbox wrapper.
104104

105105
**Not yet real**
106-
- No CI on pull requests or `main` — only tag pushes. Merged PRs #4–#8 had no
107-
automated gate.
106+
- ~~No CI on pull requests or `main`~~ — PR CI landed (`.github/workflows/ci.yml`,
107+
backlog §D1/§D3). Correctness fixes in §A are still outstanding, and the
108+
macOS E2E is nightly/label-gated rather than a per-PR gate.
108109
- No signing/notarization, no Homebrew/apt/npm, no published Docker image, no
109110
checksums on release assets. "Install" means "download from GitHub
110111
Releases and click through a Gatekeeper warning".
@@ -186,8 +187,8 @@ root.
186187
The project cannot absorb contributors or agents while merges are ungated and
187188
known races exist.
188189

189-
- PR/`main` CI: protocol tests, `agent-runtime.test.mjs`, Linux Docker E2E on
190-
every PR; macOS E2E on a schedule or label (backlog §D).
190+
- ~~PR/`main` CI: protocol tests, `agent-runtime.test.mjs`, Linux Docker E2E on
191+
every PR; macOS E2E on a schedule or label (backlog §D1, §D3).~~ **Done.**
191192
- Fix the correctness bugs that can crash or corrupt a running host: the
192193
shutdown data race on `LinuxBrowserHost`, force-unwraps in `visual compare`,
193194
the oversized `qa report`/`artifact.list` responses vs the 1 MiB frame,

‎docs/roadmap/improvements-backlog.md‎

Lines changed: 17 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -205,13 +205,20 @@ response-body inspection stays denied (P1.md:134) unless a gated design lands.
205205

206206
## §D — CI & testing (Phase 1)
207207

208-
**D1. PR/`main` CI is absent** — only tag-triggered release exists
209-
(`.github/workflows/release.yml`; PRs #4–#8 merged ungated). Add a `ci.yml`:
210-
protocol suite + `pnpm test:runtime` + `linux-docker.sh` on every PR (Linux
211-
runners); macOS build + `test.sh` on PR, full `macos-e2e.sh` nightly/label
212-
(it mutates user defaults and needs a GUI session — see
213-
`Tests/macos-e2e.sh:26-40`). Reuse the same scripts as release (keep that
214-
property).
208+
**D1. PR/`main` CI** — ~~absent; only tag-triggered release existed
209+
(`.github/workflows/release.yml`; PRs #4–#8 merged ungated).~~ **Done:**
210+
`.github/workflows/ci.yml` runs on every PR and `main` push — static checks
211+
(shell syntax across sh/bash/zsh, `docs/qa/evidence` checksum verification,
212+
whitespace), `pnpm test:runtime`, the protocol suite on Linux
213+
(`swift:6.1-bookworm` container) and macOS, and `linux-docker.sh` with the QA
214+
evidence uploaded as an artifact. `macos-e2e.sh` is gated to nightly cron,
215+
`workflow_dispatch`, or the `macos-e2e` PR label because it needs a GUI
216+
session and mutates user defaults (`Tests/macos-e2e.sh:26-40`). All jobs
217+
invoke the same scripts a developer runs locally, matching release.yml.
218+
219+
Follow-ups: mark the jobs required in branch protection (repo setting, not
220+
code); consider arm64 Linux E2E on PRs (release covers it on tags); revisit
221+
whether the macOS E2E can become a per-PR gate once runtime is measured.
215222

216223
**D2. Known test gaps (from code audit).** No tests for: peer-UID rejection
217224
(A9), MCP (C3), `Recording.swift` (ffmpeg args per format/quality, discovery
@@ -228,7 +235,9 @@ flow start-stop/network emulate/mock clear/session ops/status/stop/start/help).
228235
Also tighten `rejectsUnexpectedRequestFields` (`ProtocolTests.swift:109-114`)
229236
which currently passes for the wrong reason.
230237

231-
**D3. Web CI:** `next build` + eslint on PR (site can break invisibly today).
238+
**D3. Web CI:** ~~`next build` + eslint on PR (site can break invisibly today).~~
239+
**Done** — the `web` job in `ci.yml` runs `pnpm --filter @headless/web lint`
240+
and `build`.
232241

233242
**D4. Cross-engine conformance runner** (architecture §14) replacing drifted
234243
hand-mirrored E2E assertions (e.g. `macos-e2e.sh:231` vs `linux-e2e.sh:157`).

0 commit comments

Comments
 (0)