Skip to content

Commit fbd4c55

Browse files
lemon-mintlemon-mint
authored
Merge pull request #1 from gosuda/feat/nextgen-rewriter
Enhance Cloudflare challenge compatibility and refactor components
2 parents a47f8bd + bdbcc86 commit fbd4c55

112 files changed

Lines changed: 25204 additions & 7402 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 70 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@ name: CI
22

33
on:
44
push:
5-
branches: [main]
65
pull_request:
76
workflow_dispatch:
87

@@ -37,8 +36,9 @@ jobs:
3736
- name: Set up Node.js
3837
uses: actions/setup-node@v4
3938
with:
40-
node-version: lts/*
39+
node-version: 24.16.0
4140
cache: npm
41+
cache-dependency-path: package-lock.json
4242
- name: Print toolchain versions
4343
run: |
4444
go version
@@ -50,6 +50,14 @@ jobs:
5050
- name: Install wasm-bindgen CLI
5151
run: cargo install wasm-bindgen-cli --version 0.2.122 --locked
5252

53+
- name: Cache Puppeteer browser
54+
uses: actions/cache@v4
55+
with:
56+
path: .puppeteer-cache
57+
key: puppeteer-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
58+
restore-keys: |
59+
puppeteer-${{ runner.os }}-
60+
5361
- name: Install Node dependencies
5462
run: npm ci
5563

@@ -59,8 +67,68 @@ jobs:
5967
- name: Run Go tests
6068
run: go test ./...
6169

70+
- name: Run Go js/wasm tests
71+
run: npm run test:wasm
72+
6273
- name: Run JavaScript and Puppeteer tests
6374
run: npm test
6475

6576
- name: Build deployable artifacts
6677
run: npm run build
78+
79+
lint:
80+
name: Lint (Go, Rust, JavaScript)
81+
runs-on: ubuntu-24.04
82+
timeout-minutes: 20
83+
84+
steps:
85+
- name: Check out repository
86+
uses: actions/checkout@v4
87+
88+
- name: Set up Go
89+
uses: actions/setup-go@v5
90+
with:
91+
go-version-file: go.mod
92+
cache: true
93+
94+
- name: Set up Rust
95+
uses: dtolnay/rust-toolchain@stable
96+
with:
97+
targets: wasm32-unknown-unknown
98+
components: clippy, rustfmt
99+
100+
- name: Set up Node.js
101+
uses: actions/setup-node@v4
102+
with:
103+
node-version: 24.16.0
104+
cache: npm
105+
cache-dependency-path: package-lock.json
106+
107+
- name: Install golangci-lint v2.12.2
108+
run: |
109+
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/v2.12.2/install.sh \
110+
| sh -s -- -b "$(go env GOPATH)/bin" v2.12.2
111+
golangci-lint version
112+
113+
- name: Verify golangci-lint config
114+
run: golangci-lint config verify
115+
116+
- name: Run golangci-lint (native)
117+
run: golangci-lint run --timeout=5m
118+
119+
- name: Run golangci-lint (js/wasm)
120+
run: GOOS=js GOARCH=wasm golangci-lint run --timeout=5m
121+
122+
- name: Check Rust formatting
123+
run: cargo fmt --manifest-path rewriter-rs/Cargo.toml --all --check
124+
125+
- name: Run clippy
126+
run: cargo clippy --manifest-path rewriter-rs/Cargo.toml --all-targets -- -D warnings
127+
128+
- name: Install Node dependencies
129+
run: npm ci
130+
env:
131+
PUPPETEER_SKIP_DOWNLOAD: "true"
132+
133+
- name: Run Biome
134+
run: npx biome ci web scripts test

‎.gitignore‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,5 +6,11 @@ web/wasm_exec.js
66
web/oxc_parser_wasm_bg.wasm
77
node_modules/
88
coverage/
9+
.cache/
10+
artifacts/
911
rewriter-rs/target/
10-
wasm-kernel
12+
/wasm-kernel
13+
GOAL.md
14+
.claude/
15+
.npm-cache
16+
.puppeteer-cache

‎.golangci.yml‎

Lines changed: 195 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,195 @@
1+
# golangci-lint v2 configuration (schema version "2").
2+
#
3+
# All enabled linters land HARD (CI must be green). This INCLUDES the
4+
# complexity gates (cyclop / gocognit / nestif): they are ACTIVE and enforcing
5+
# — new code over budget fails CI. See linters.settings for the thresholds and
6+
# linters.exclusions for the test-file carve-out.
7+
#
8+
# A few pre-existing, *intrinsic* findings are narrowly excluded with a
9+
# "# TODO(ratchet):" comment (e.g. SHA-1 mandated by the RFC6455 WebSocket
10+
# handshake, protocol byte encodings, err-shadowing). The only remaining
11+
# complexity suppressions are narrow inline //nolint:<linter> // TODO(complexity)
12+
# at the few wasm-tagged (js && wasm) protocol/membrane sites; every other
13+
# function is decomposed under budget.
14+
version: "2"
15+
16+
run:
17+
timeout: 5m
18+
# Lint test files too. The js/wasm pass (GOOS=js GOARCH=wasm) is invoked
19+
# separately and is the only pass that covers the //go:build js && wasm
20+
# files: bridge_js.go / conn_js.go / cmd/wasm-kernel/main.go.
21+
tests: true
22+
23+
linters:
24+
# Start from an empty set so the enabled linters are exactly the spec:
25+
# this guarantees no complexity linter is silently active.
26+
default: none
27+
enable:
28+
- govet
29+
- staticcheck
30+
- errcheck
31+
- ineffassign
32+
- unused
33+
- unparam
34+
- unconvert
35+
- misspell
36+
- gosec
37+
# A.2: complexity gates flipped to HARD error (see settings + exclusions).
38+
- cyclop
39+
- gocognit
40+
- nestif
41+
42+
settings:
43+
govet:
44+
# Enable every vet analyzer except fieldalignment (too noisy / churny).
45+
enable-all: true
46+
disable:
47+
- fieldalignment
48+
# TODO(ratchet): `shadow` flags idiomatic `if _, err := ...` blocks
49+
# across app and test code. Satisfying it requires renaming variables
50+
# in application logic, which A.1 must not touch. Re-enable after a
51+
# dedicated shadow-cleanup pass. Reported as a concern.
52+
- shadow
53+
misspell:
54+
locale: US
55+
# TODO(ratchet): "cancelled" (British spelling) appears as a local
56+
# variable in security-membrane code (internal/swhttp/bridge_js.go).
57+
# A.1 must not reformat/edit the membrane. Ignore the word here rather
58+
# than rename the variable. Reported as a concern.
59+
# (v2 schema: misspell uses `ignore-rules`, not the v1 `ignore-words`.)
60+
ignore-rules:
61+
- cancelled
62+
# NOTE: staticcheck is intentionally left at its golangci-lint default
63+
# check set (which already excludes the ST10xx stylecheck rules). Do NOT
64+
# add `checks: [all, ...]` here: that switches the entire ST family on and
65+
# surfaces unrelated structural findings (e.g. ST1000 package comments).
66+
# QF1003 is deferred via linters.exclusions.rules below instead.
67+
errcheck:
68+
# TODO(ratchet): unchecked errors on best-effort write paths.
69+
# `out` is a *bufio.Writer; WriteString on it can only fail if the
70+
# underlying writer errors, and an immediate Flush already surfaces that.
71+
# Fixing the remaining sites edits application logic (forbidden in A.1).
72+
# Reported as a concern. (Deferred Close/SetDeadline are covered by the
73+
# std-error-handling preset and the SetDeadline rule below.)
74+
exclude-functions:
75+
- (*bufio.Writer).WriteString
76+
gosec:
77+
# TODO(ratchet): the findings below are INTRINSIC to a TLS-intercepting
78+
# proxy / SOCKS5 / WebSocket protocol implementation and cannot be
79+
# "fixed" without changing security-membrane behavior (forbidden in A.1).
80+
# Each excluded sub-rule is reported as a concern for the A.2 ratchet:
81+
# G101 - false positive: "zp-streamiso-v1\x00" is a protocol prefix,
82+
# not a credential.
83+
# G114 - http.ListenAndServe in the dev server (server hardening is a
84+
# separate task).
85+
# G115 - int->byte/uint16 conversions are deliberate protocol-frame
86+
# length/port encodings (SOCKS5 / WS).
87+
# G124 - cookie jar mirrors upstream Set-Cookie attributes verbatim by
88+
# design; it must not inject Secure/HttpOnly/SameSite.
89+
# G304/G703 - os.Open of an operator-supplied path (config/asset
90+
# loading); both fire on the same call site.
91+
# G401/G505 - SHA-1 is MANDATED by the RFC6455 WebSocket handshake.
92+
# G710 - http.Redirect target is policy-validated upstream.
93+
#
94+
# G104 is a different class: it is gosec's GENERIC unchecked-error rule,
95+
# fully redundant with errcheck (which stays enabled globally as the
96+
# authoritative, more configurable unchecked-error linter). Excluding
97+
# G104 removes double-reporting on lines where errcheck is deliberately
98+
# excluded; it does NOT reduce unchecked-error coverage.
99+
excludes:
100+
- G101
101+
- G104
102+
- G114
103+
- G115
104+
- G124
105+
- G304
106+
- G401
107+
- G505
108+
- G703
109+
- G710
110+
111+
# ----------------------------------------------------------------------
112+
# A.2: complexity gates are now HARD errors. New code over budget fails CI.
113+
# Pre-existing residuals are handled HONESTLY: _test.go is excluded below
114+
# (test-function complexity is out of scope), and the remaining wasm-tagged
115+
# (js && wasm) protocol/membrane functions still over budget carry a narrow
116+
# inline `//nolint:<linter> // TODO(complexity): ...` at each site. Native
117+
# offenders have been decomposed under budget. cyclop.package-average is
118+
# intentionally omitted (fragile).
119+
cyclop:
120+
max-complexity: 10
121+
gocognit:
122+
min-complexity: 15
123+
nestif:
124+
min-complexity: 4
125+
# ----------------------------------------------------------------------
126+
127+
exclusions:
128+
# Be lax on generated files (e.g. files with a generated-code header).
129+
generated: lax
130+
# Opt into golangci-lint's built-in "std-error-handling" preset (the old
131+
# EXC0001): excludes unchecked errors from best-effort cleanup calls such
132+
# as deferred Close/Flush. v2 ships NO default exclusions, so this is an
133+
# explicit, narrow opt-in rather than a blanket relaxation.
134+
presets:
135+
- std-error-handling
136+
# Skip vendored / build-output / non-Go trees entirely.
137+
paths:
138+
- dist
139+
- bin
140+
- rewriter-rs/target
141+
- node_modules
142+
rules:
143+
# Test files: relax rules that are noisy or low-value in tests.
144+
# A.2: cyclop/gocognit/nestif are excluded here too — test-function
145+
# complexity is OUT OF SCOPE (e.g. table-driven / scenario bodies like
146+
# relay_test.go TestBridgeInternalSOCKS, jar_test, transform_*_test).
147+
# Production complexity stays HARD.
148+
- path: _test.go
149+
linters:
150+
- gosec
151+
- errcheck
152+
- unparam
153+
- cyclop
154+
- gocognit
155+
- nestif
156+
# TODO(ratchet): QF1003 ("use tagged switch") is a stylistic suggestion;
157+
# acting on it edits application logic. Deferred. Reported as a concern.
158+
- linters:
159+
- staticcheck
160+
text: "QF1003"
161+
# TODO(ratchet): deferred SetDeadline reset on a connection (best-effort
162+
# cleanup) in the SOCKS5 client. The receiver is an anonymous interface,
163+
# so it is excluded by path+text here rather than via
164+
# errcheck.exclude-functions. Scoped to this one call site. Reported as
165+
# a concern.
166+
- path: internal/socks5/client\.go
167+
linters:
168+
- errcheck
169+
text: "c\\.SetDeadline"
170+
# TODO(ratchet): in-flight scaffolding on the feature branch. These
171+
# specific symbols/params are wired for the nextgen rewriter and are not
172+
# safe to delete in a config-only task. Scoped by name so genuinely dead
173+
# code added later is still caught. Reported as a concern; revisit in A.2.
174+
- path: internal/htmltx/transform\.go
175+
linters:
176+
- unused
177+
text: "rewriteEventHandler|pathEscape"
178+
- path: internal/htmltx/transform\.go
179+
linters:
180+
- unparam
181+
text: "wrapAttrURL - nav is unused"
182+
- path: cmd/zeroproxy-server/main\.go
183+
text: "workerBootstrap - r is unused"
184+
linters:
185+
- unparam
186+
187+
formatters:
188+
enable:
189+
- gofumpt
190+
exclusions:
191+
paths:
192+
- dist
193+
- bin
194+
- rewriter-rs/target
195+
- node_modules

‎.npmrc‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
cache=.npm-cache

‎.puppeteerrc.cjs‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
const path = require('node:path');
2+
3+
module.exports = {
4+
cacheDirectory: path.join(__dirname, '.puppeteer-cache'),
5+
};

‎AGENTS.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# AGENTS.md — ZeroProxy
2+
3+
ZeroProxy is a **human-in-the-loop** virtual-browsing privacy membrane: a real person drives a real browser, and target traffic egresses only through `Service Worker → Go WASM kernel → WebSocket/yamux → SOCKS5 → uTLS`. `ARCHITECTURE.md` holds the data-flow diagram and the full **Core invariants** list — read it before touching membrane/transport code; this file only adds what that doesn't, the conventions and traps that are expensive to rediscover.
4+
5+
## Membrane/protocol refactor discipline (load-bearing)
6+
7+
- A behavior-preserving change to membrane or protocol code must be proven by a **transient differential harness**, not a green suite: freeze the pre-change function verbatim under a new name, drive both old and new over a generated + edge corpus through the package's existing test seam (`scriptedRW` in socks5, `net.Pipe`/`pipeMux` in wsproto/zphttp), assert **0 mismatches** (return value, error string, bytes on the wire), then **delete the harness — never commit it** (`zz_*` scaffolding is correctly rejected in review). Keep a *permanent* characterization/adversarial oracle. *Why:* the `transform.go` decomposition passed the full suite but silently changed a marker; only a differential caught it. Suite-green ≠ behavior-preserved.
8+
9+
- Removing a complexity `//nolint` is only real if the gate actually fires on that file. Prove it **red-before**, not just green-after: drop the pre-decomposition original (nolint stripped) at the path, confirm golangci **fails** on the complexity linter, then restore the decomposed file byte-identical (md5). *Why:* a stale `.golangci.yml` header once claimed the gates were "disabled" while they were live — green-after alone would have been hollow.
10+
11+
## Lint / complexity gates
12+
13+
- Complexity gates are **live and hard**: golangci `cyclop` ≤10 / `gocognit` ≤15 / `nestif` ≤4 (`_test.go` excluded), clippy `cognitive_complexity = "deny"` @15, Biome `noExcessiveCognitiveComplexity` @15. **Decompose to satisfy them — do not add new suppressions.** *Why:* the campaign is burning these down, not accumulating them.
14+
- Known, deliberate remaining suppressions (a burn-down tail, not free license) take three forms — keep them straight, they are easy to conflate, and verify against `git grep` not this list, which drifts as the burn-down lands: **(1)** inline `//nolint:cyclop // TODO(complexity)` on exactly one wasm-tagged kernel function — `cmd/wasm-kernel/main.go` `relayEnsure` (cyclop 11; its only safe decomposition splits the engine mutex region and is not differentially verifiable without live `wsconn.Dial`); **(2)** a Biome glob override turning `noExcessiveCognitiveComplexity` **off** for exactly two files — `web/runtime-prelude.js` (the ~4.4k-line membrane, ~72 fns over budget — the one genuinely large remaining decomposition workstream) and `web/index.html` (its inline bootstrap) — plus `test/**` (test bodies out of scope); **(3)** a single inline `biome-ignore lint/complexity/noExcessiveCognitiveComplexity` at `web/worker-prelude.js` (module IIFE — no inner fn exceeds 15; the count is the wrapper-guard aggregate, so splitting relocates the global-exposure boundary rather than cutting complexity). `web/zp-core.js` carries **no** inline complexity ignores (both decomposed). The separate `web/**` Biome override disables only the **formatter** (so `biome ci` never reformats the membrane), **not** the complexity linter — the cognitive-complexity gate stays **hard on every other `web/` file, including `sw.js`**. All of these need a differential-harness decomposition, not a quick edit.
15+
16+
## Build / verify traps
17+
18+
- **wasm-tagged files** (`//go:build js && wasm`: `cmd/wasm-kernel/main.go`, `internal/swhttp/bridge_js.go`, `internal/wsconn/conn_js.go`) are **skipped by `go test ./...` and the native golangci pass.** Lint/build coverage is `GOOS=js GOARCH=wasm golangci-lint run` and `GOOS=js GOARCH=wasm go build ./cmd/wasm-kernel` (`npm run lint:go` runs both golangci passes). Wasm-tagged **tests** (e.g. `internal/swhttp/bridge_js_test.go`, `cmd/wasm-kernel/wsstream_test.go`) are likewise skipped by `go test ./...`; they run only under `npm run test:wasm`, which executes them via the Go `go_js_wasm_exec` runner (CI runs this as its own step). The script wraps the run in `env -i` preserving only `PATH`/`HOME`/`GOCACHE`/`GOMODCACHE` — the wasm runtime copies the whole env into a bounded argv+env buffer, so an unstripped (large) env overflows it (`total length of command line and environment variables exceeds limit`). Run `npm run test:wasm` after any transport/bridge change or you have verified nothing for that code.
19+
- Run `golangci-lint cache clean` before trusting lint results — the results cache serves stale issues from deleted worktrees (paths like `../../../../tmp/…`, "can't read file").
20+
- Use the npm test scripts (`npm test` / `test:js` / `test:e2e` → `node scripts/test.mjs [js|e2e]`). **Do not** run `node --test test/js` — Node 24 treats the directory as a module and reports a spurious failure.
21+
- Rust rewriter behavior is not covered by `npm test`: run `cargo test --manifest-path rewriter-rs/Cargo.toml` after rewriter changes. `npm run lint:rust` is clippy + fmt only, and CI runs the Rust tests as a separate gate. *Why:* the Rust WASM rewriter is the static compiler pipeline for target scripts/CSS, so a green JS/Puppeteer suite alone can miss parser/rewriter regressions.
22+
- Do not blanket-format `web/**`. Biome formatting is intentionally disabled there, and `npm run fmt:fix` formats Go/Rust plus `scripts`/`test`, not the membrane web assets. *Why:* large membrane files keep reviewable hand-shaped layout until a differential-harness decomposition proves the behavior-preserving change.
23+
- **E2E flake:** the two heavy Puppeteer tests can mutually starve under load — one times out at the ~31.5s page deadline while the other passes, and *which* one fails migrates between runs. A migrating failure is environmental, not a regression (a real regression fails the same test deterministically); re-run, or run the e2e tests individually, before blaming a code change.
24+
25+
## Commits
26+
27+
- Conventional Commits, one concern per commit; substantive commits carry an `Op: compress|extend|correct` trailer (plus `Restores: …` for `correct`). Use the configured git identity — **no** `--author`, `Co-Authored-By`, `Signed-off-by`, or any agent trailer; do not mutate git config.

0 commit comments

Comments
 (0)