Enhance Cloudflare challenge compatibility and refactor components - #1
Merged
Conversation
…ove storage/cookie reconciliation
…mpatibility, and dynamic execution support
…d redirect policies, and improved response metadata tracking
…ution mechanism for iframes
…NamedNodeMap methods and improving SVG fragment handling
…ags while making constructor descriptors writable
…script nonce preservation
…to remove Cloudflare-specific frame restrictions.
…nd remove Cloudflare bypass
…ough standardized navigator identities, client hint normalization, and robust object definition shims.
Apply rustfmt to rewriter-rs/src/lib.rs and fix all default-clippy findings under `-D warnings` (manual_flatten x3, unnecessary_to_owned x2, manual_contains x2). Add rewriter-rs/clippy.toml setting cognitive-complexity-threshold = 25; cognitive_complexity is an allow-by-default nursery lint so it stays inert under `-D warnings` (authored-but-disabled for Workstream A.1). The clippy.toml comment documents the planned 25 -> 20 -> 15 ratchet for Workstream A.2. Op: extend
Add @biomejs/biome@^2 (resolved 2.4.16) as a dev dependency and a schema-valid Biome 2.x config. `npx biome ci web scripts test` exits 0 on the current code; the security-membrane files (web/runtime-prelude.js, web/sw.js, web/worker-prelude.js) are not reformatted. Config file is biome.jsonc (not biome.json): Biome escalates lint diagnostics from warning/info to error severity when a strict .json config carries comments, which flips `biome ci` red. .jsonc is a first-class Biome config filename and lets each disabled rule carry an inline, greppable TODO(ratchet) marker for the A.2 burn-down to find. Formatter: 2-space indent, line width 100, single quotes. Enforced on scripts/ and test/ (those 7 files reformatted; whitespace/wrapping only, no logic change, JS unit tests still pass). web/** has the formatter disabled via override so membrane files are never reformatted. Linter: recommended ruleset. Each rule below is disabled now and deferred to Workstream A.2 with an inline TODO(ratchet) comment at its line: - complexity/noExcessiveCognitiveComplexity: OFF by A.1 design, deferred to A.2 after the complexity burn-down (cognitive gate). - correctness/noInnerDeclarations: ~16 violations in web/runtime-prelude.js (membrane; widespread). - suspicious/noAssignInExpressions: 5 violations across web/runtime-prelude.js (membrane) and test/e2e/proxy.test.js. - suspicious/noShadowRestrictedNames: 4 violations in membrane runtime-prelude.js/worker-prelude.js (intentional toString masking). - suspicious/noRedeclare: 1 violation, membrane web/runtime-prelude.js. - suspicious/useIterableCallbackReturn: 1 violation, membrane web/runtime-prelude.js. - a11y/useButtonType: 1 violation, web/index.html (membrane HTML). The last three are single violations disabled because they live in membrane files this workstream must not edit, not because they are widespread. Globals: ZP + Go declared globally; Node globals added for test/** and scripts/**. Browser/serviceworker/webworker identifiers are NOT enumerated (Biome 2.x has no ESLint-style env presets, and noUndeclaredVariables is not in the recommended set so they cannot fire) — a documented limitation to revisit if A.2 enables noUndeclaredVariables. assist/organizeImports is off to keep CI green without source edits. Ignores: dist, node_modules, bin, rewriter-rs/target, package-lock.json, and generated wasm glue (.gitignore is also honored). Op: extend
Characterization tests pinning the membrane's observable contract so the
upcoming aggressive complexity burn-down cannot silently weaken a security
invariant. Each pin is mutation-proven non-vacuous (deleting the guarded
branch turns the suite red).
JS (test/js/membrane-invariants.test.js): no-direct-egress / fail-closed
classification (never falls back to native fetch(event.request)), masking-hook
presence (ZP artifacts filtered from enumeration), CSP default-deny.
Go (internal/{headers,zphttp,cookiejar}/*_freeze_test.go): ConstructorPolicy
strip-set with COOP/COEP/CORP passthrough, cross-origin cookie-injection guard,
host-only cookie scope, negative-Max-Age deletion, referer https->http suppression.
Op: correct
Restores: spec:membrane-invariants
Increment 1 of Cloudflare challenge compatibility (compatibility, NOT bypass/solver/token-synthesis). Adds a header+URL-only classifier and a default-OFF, per-tab arm opt-in that, when BOTH signals are present, emits the internal X-ZP-Challenge-Compat marker via the existing X-ZP-* mechanism. No CSP projection, no egress capability, no eval manufacture yet: this increment is INERT unless a tab is explicitly armed AND the response is classified as a challenge document/subresource. cmd/wasm-kernel/challenge.go (untagged sibling so native `go test ./...` actually compiles+runs it alongside main_stub.go): - targetIsChallengeDocument(header, finalURL): HEADER+URL ONLY, never reads the body. Keys on cf-mitigated:challenge, exact host challenges.cloudflare.com (not suffix), or path prefix /cdn-cgi/challenge-platform/. Flat (cyclo 5). - applyChallengeCompat(header, armed, finalURL): two-signal gate; sets the marker only when armed AND classified; nil-safe; INERT otherwise. Flat (cyclo 4). internal/zphttp/roundtrip.go: ChallengeCompat bool on TabState, set once at tab creation under k.mu and never mutated, so the lock-free read in jsHTTP is race-free by construction (publish-under-lock happens-before). cmd/wasm-kernel/main.go: tabFromValues gains a challengeCompat arg sourced from the internal X-Zp-Challenge-Compat-Arm request header (nothing sends it in B1, so the OFF path is byte-identical). jsHTTP delegates the armed branch to the flat helper (its complexity is unchanged). cookie/stream callers pass false. .gitignore: anchor the stale `wasm-kernel` ignore to `/wasm-kernel` so package source under cmd/wasm-kernel/ is no longer silently swallowed (real artifact is dist/kernel.wasm, already covered). Verification: go test ./... (75 pass) + new gate-matrix test under -race; frozen invariants green (Go 37 + node 24); golangci native + GOOS=js GOARCH=wasm clean; npm run build green. Review fix (amend, comment-only, behavior-preserving): the reviewer flagged that the inbound X-Zp-Challenge-Compat-Arm reader is a page-influenceable request header with no inbound strip and no forward-obligation comment, asymmetric to the documented OUTBOUND marker strip in challenge.go. In B1 this is doubly inert (birth-only semantics drop a forged arm on an already-born tab; no marker consumer exists), so the fix is documentation, not code (adding the strip now would be B4 work and break minimal-change / byte-identical-OFF): - main.go tabFor: added a B4 INBOUND-STRIP OBLIGATION comment mirroring the outbound B4 STRIP OBLIGATION in challenge.go. It records that the arm header is page-forgeable and that when B4 lands the trusted arm sender, web/sw.js transportFetch and web/runtime-prelude.js fetchThroughRuntime must authoritatively set/delete it the SAME way they handle X-ZP-Tab-Id / X-ZP-Runtime-Token, so a proxied page can never supply it. - main.go tabFromValues: strengthened the existing-tab early-return into a stated SECURITY INVARIANT (birth-only arm). ChallengeCompat is set ONLY at tab birth because the arm header is page-forgeable; honoring it on the existing-tab path would convert the forgeable inbound header into an active self-arm primitive. Do NOT re-arm a live tab. - roundtrip.go TabState.ChallengeCompat: rewrote the field comment from a race/concurrency note into the load-bearing SECURITY INVARIANT (set-once for the security property, race-freedom is a side benefit). The diff is purely additive comments (no executable line changed), so the OFF and non-challenge paths stay byte-identical and frozen invariants stay green. Op: extend
Increment 2 of Cloudflare challenge compatibility (compatibility, NOT
bypass/solver/token-synthesis). ConstructorPolicy gains a caller-computed
challengeCompat bool that, when true, SKIPS only the Cache-Control: no-store
overwrite so Cloudflare's own cache/update semantics for its challenge
SUBRESOURCES (e.g. turnstile api.js) survive. Everything else -- the full
hidden-header strip, the CORS emulation, nosniff, and the proxy transport --
is untouched, so it grants no egress and manufactures no eval. When false
(every existing call path) the no-store overwrite is applied exactly as
before; the default/OFF path stays behaviorally identical and the frozen
policy invariants stay green.
internal/headers/policy.go: ConstructorPolicy signature gains challengeCompat;
the unconditional dst.Set("Cache-Control","no-store") becomes guarded by
`if !challengeCompat`. This is the third caller-computed bool in the same
pattern as bodyTransformed/bodyDecoded -- the function does not classify; the
document-vs-subresource decision lives in the caller (layering: the classifier
targetIsChallengeDocument / isDocumentRequest are package main and cannot be
imported by internal/headers).
cmd/wasm-kernel/challenge.go: new flat helper challengeSubresourceSkip(armed,
isDoc, header, finalURL) is the two-signal gate -- returns true ONLY when armed
AND NOT a document navigation AND classified as a challenge. The isDoc==false
term is security-load-bearing: it keeps the challenge DOCUMENT (navigation
HTML) on no-store and lets ONLY subresources preserve Cloudflare's cache
semantics. Lives in the untagged, natively-testable sibling (like B1's
applyChallengeCompat) so the discrimination is unit-tested at the layer where
it is actually decided, not just by inspection.
cmd/wasm-kernel/main.go: jsHTTP computes the gate via challengeSubresourceSkip
on the RAW target header/URL before policy construction, and feeds the SAME
bool to BOTH ConstructorPolicy applications. ConstructorPolicy runs twice on
the same header in this path (directly here, then again inside
swhttp.ResponseToJS); threading one shared bool to both is required, else the
second pass would silently re-impose no-store and the skip would no-op. Default
OFF: nothing arms a tab yet (B1), so the gate is always false and the path is
byte-identical.
internal/swhttp/bridge_js.go + bridge_stub.go: ResponseToJS threads the new
bool through to its ConstructorPolicy call (both the js/wasm impl and the
native stub, kept signature-parallel so native `go test ./...` compiles).
Tests: policy_test.go adds TestConstructorPolicyChallengeCompatSkipsNoStore --
compat=true preserves a present target Cache-Control AND leaves a header-less
response header-less (no synthesized no-store); compat=false keeps no-store
(the challenge DOCUMENT / default path). challenge_test.go adds
TestChallengeSubresourceSkip -- pins the isDoc guard so an armed, classified
DOCUMENT returns false (keeps no-store) while an armed, classified SUBRESOURCE
returns true. The existing strip test, the freeze tests, and bridge_js_test.go
are updated to pass the new bool (false) so the default-path assertions are
unchanged.
Verification: go test ./... (83 pass) native; GOOS=js GOARCH=wasm build of
./cmd/wasm-kernel + vet clean; internal/swhttp js/wasm test green under
go_js_wasm_exec; frozen invariants green (Go 38 + node 24); golangci native +
GOOS=js GOARCH=wasm clean on changed packages; npm run build + test:js (53) +
biome ci (exit 0) green.
Op: extend
Increment 3 of Cloudflare challenge compatibility (compatibility, NOT bypass/solver/token-synthesis). web/zp-core.js fixedCSP() gains an opt-in options.challengeCompat that, when ON, PROJECTS the challenge execution model so a REAL human's Cloudflare challenge can run inside the membrane: script-src / connect-src / frame-src / child-src ADD https://challenges.cloudflare.com (connect-src via the deduped Set, so 'self' is already present). frame/child/worker keep their existing blob: capability -- the spec's "worker-src adds blob:" is a no-op (blob: is already in worker-src), so worker-src is left untouched. The projection is purely ADDITIVE around the unchanged script ternary and directive literals. Honor-not-manufacture eval (F3): 'unsafe-eval' is NOT emitted by challengeCompat. It still rides ONLY the pre-existing allowDynamicCompile branch, which the kernel sets target-authoritatively (targetDynamicCompileAllowed -> cspPolicyAllowsEval over the TARGET CSP). If the challenge's own CSP did not grant eval, neither do we. No egress escape: the projection adds a single fixed host token; it adds NO wildcard (no connect-src */script-src *) and NO direct-network capability. Challenge fetches still route through the proxy transport (/zp/api/*) -- this CSP only governs which sources the document may name, not the transport. Default OFF / byte-identical: challengeCompat defaults false; when false (or absent) cf == "" and no Set add, so the emitted CSP is byte-identical to today. fixedCSP([], {}) === fixedCSP() is asserted, and the source-text pins in static-policy.test.js (the script-src literal, the connect-src/script-src no-wildcard guards) are unchanged. The function stays flat (one if, one ternary). This increment is INERT in production: nothing wires X-ZP-Challenge-Compat into web/sw.js addCSP yet (consistent with B1/B2 staying inert until the consumer-wiring increment). The tests exercise fixedCSP directly, so the projection is verified at its decision point without activating it. test/js/membrane-invariants.test.js (Invariant 3b) adds three cases: - OFF path byte-identity: fixedCSP([], {}) and {challengeCompat:false} both === the default CSP string. - Armed delta proof: parse both CSPs to ORDERED directive entries (not a name-keyed map, so a smuggled duplicate directive cannot hide), assert the directive sequence is identical and that the ONLY added token is exactly https://challenges.cloudflare.com on script/connect/frame/child and nothing elsewhere; no baseline token dropped; no bare wildcard on any execution/egress-capable directive (script/connect/frame/child/ worker/object-src). This pins "adds EXACTLY the challenge host", closing the gap a mere "no *" check would leave (https:/extra-host would slip). - F3 eval discriminator: armed WITHOUT allowDynamicCompile has no bare 'unsafe-eval'; armed WITH it does -- proving honor-not-manufacture. Verification: npm run build green; npm run test:js 56 pass (was 53); node --test frozen invariants (membrane + static-policy) 27 pass; Go frozen membrane invariants (headers/zphttp/cookiejar) 38 pass; e2e 1 pass; biome ci web scripts test exit 0 (no new findings -- my additions add zero biome warnings/errors). Op: extend
Increment 4 of Cloudflare challenge compatibility (compatibility, NOT bypass/solver/token-synthesis). The service worker now consumes the kernel's internal X-ZP-Challenge-Compat response marker and projects the challenge CSP, while plumbing the per-tab arm opt-in through the tab-open path. Default OFF and the OFF/non-challenge path stays byte-identical (frozen invariants green). web/sw.js: - addCSP (document/script-document path): reads AND deletes the kernel's X-ZP-Challenge-Compat marker, then passes challengeCompat into ZP.fixedCSP. The marker is the kernel's two-signal gate output (armed AND header/URL classified), so the document CSP is two-signal by construction. The delete is load-bearing: the internal marker must NEVER reach the proxied page. - isChallengeURL: flat URL-only classifier mirroring the kernel's targetIsChallengeDocument (host == challenges.cloudflare.com or path prefix /cdn-cgi/challenge-platform/), try/catch -> false. Header/body never read. - rewriteScriptResponse / scriptResponseHeaders: the script/worker CSP projection is now TWO-SIGNAL. rewriteScriptResponse gates on BOTH the per-tab arm bit (opt.challengeCompat) AND isChallengeURL(opt.targetUrl), so a non-challenge script/worker on an armed tab stays byte-identical to the default CSP (fixes the reviewer's single-signal finding). addCSP already deletes the marker upstream on every response; scriptResponseHeaders also deletes X-ZP-Challenge-Compat as defense-in-depth so it can never leak on the script path. The projection adds EXACTLY the challenge host (frozen invariant), no egress, no eval (worker-src stays 'self' blob:); honor-not-manufacture is preserved (unsafe-eval rides only the existing allowDynamicCompile grant). All 3 call sites (virtualSubresource, /zp/api/script, /zp/api/worker-script) thread the bit. - transportFetch: authoritatively delete-then-conditionally-set the kernel arm header X-Zp-Challenge-Compat-Arm the SAME way as X-ZP-Tab-Id / X-ZP-Runtime- Token (per B1's INBOUND-STRIP OBLIGATION). The unconditional delete drops any page-forged value (e.g. via a /zp/api/fetch payload header); the conditional set re-adds it ONLY for an armed tab. A proxied page can never self-arm. - createTab / ZP_OPEN_SHARE: challengeCompat is the per-tab arm bit, set ONCE at tab birth from the explicit opt-in (default OFF), mirroring the kernel's birth-only TabState.ChallengeCompat so a live tab can never be re-armed. test/js/challenge-compat-sw.test.js: behavioral tests (zp-core + sw.js loaded into one vm, same harness as membrane-invariants) assert armed+marker -> CSP includes the challenge host AND the internal marker is stripped; unmarked -> byte-identical default CSP with no challenge host; honor-not-manufacture eval; the two-signal script gate (armed + non-challenge URL -> byte-identical default CSP; armed + challenges.cloudflare.com or /cdn-cgi/challenge-platform/ URL -> challenge host present; unarmed + challenge URL -> default CSP); isChallengeURL URL-only classification incl. malformed-URL false; the marker is stripped from script response headers; createTab defaults OFF; and static guards that all 3 rewriteScriptResponse sites thread the bit and transportFetch strips/sets the arm header. Verification: npm run build green; npm run test:js 68 pass; frozen invariants (membrane-invariants + static-policy) 27 pass; biome ci web scripts test exit 0. Op: extend
Increment 5 (final activation) of Cloudflare challenge compatibility
(compatibility, NOT bypass/solver/token-synthesis). B1-B4 built the dormant
mechanism (kernel classifier + birth-only per-tab arm, no-store skip, fixedCSP
projection, sw.js marker threading), but index.html never wired the arm, so a
human solving a real Turnstile challenge in their own browser could not opt in.
This makes the arm USER-ACTIVATABLE through the TRUSTED window->SW->kernel hop
ONLY. Default OFF: unchecked => no arm header => byte-identical behavior.
web/index.html:
- Minimal opt-in control: an unchecked checkbox "Challenge compatibility mode
(Cloudflare Turnstile)" near the URL form. Wired inside the existing
nonce="zp" script block (no inline handler; CSP unchanged). The form's URL
input is byte-identical to before (validation contract unchanged); the flex
rule is rescoped input -> #url so the new checkbox is not stretched.
- openTarget threads document.getElementById('challenge-compat').checked into
the window->SW ZP_OPEN_SHARE message. This is the ONLY user surface for the
arm. The cold direct-link path (handleShare) has no checkbox and stays
unarmed by design: a shared /zp/p link can never silently arm someone's tab.
(The task's "zp-core.js threads the boolean" attribution is loose; the
ZP_OPEN_SHARE literal is built in index.html, so the thread lives there.)
web/runtime-prelude.js:
- fetchThroughRuntime strips any inbound X-Zp-Challenge-Compat-Arm (defense in
depth), mirroring the X-ZP-Tab-Id / X-ZP-Runtime-Token handling, per B1's
INBOUND-STRIP OBLIGATION comment. A proxied page can never supply the arm.
The sw.js trusted-hop plumbing (createTab arm param, ZP_OPEN_SHARE->createTab,
and transportFetch's authoritative DELETE-then-conditional-SET of
X-Zp-Challenge-Compat-Arm) was pre-landed by the prior increment and is left
intact and verified by the existing B4 plumbing guard; no sw.js diff here.
test/js/challenge-compat-sw.test.js:
- B5 behavioral arm tests run REAL transportFetch (initKernel stubbed past the
readiness gate; __go_jshttp captures the final kernel request): ARMED tab ->
X-Zp-Challenge-Compat-Arm:1 set from trusted per-tab state; UNARMED tab -> no
arm header (OFF path); page-FORGED arm header on an UNARMED tab is DELETED
(forgery blocked); ARMED tab overrides a forged value with the trusted :1.
- runtime-prelude strip (text-level) + index.html opt-in wiring/default-off
guards.
Two-signal gate preserved (arm AND header/URL classification); no egress
escape; honor-not-manufacture eval; internal X-ZP-Challenge-Compat marker never
leaks. OFF/non-challenge path byte-identical (frozen invariants green).
Verification: npm run build green; npm run test:js 75 pass; npm run test:e2e 1
pass; go test ./... green; frozen invariants (Go headers/zphttp/cookiejar 38 +
JS membrane-invariants/static-policy 27) green; golangci native + GOOS=js
GOARCH=wasm 0 issues; clippy 0 issues; biome ci web scripts test exit 0.
Op: extend
Task B6: validate the dormant-no-more Cloudflare challenge COMPATIBILITY mechanism (B1-B5) end-to-end in a real browser, against a LOCAL fixture that mimics a challenge WITHOUT contacting Cloudflare. This is compatibility, never a solver/forgery/bypass; real Turnstile clearance stays a human-run live smoke test. test/e2e/turnstile-compat.test.js (new): - Local challenge fixture clones the proxy.test.js scaffolding (build dist, spawn zeroproxy-server with -socks internal, Puppeteer with proxy.localhost host-resolver mapping). The fixture serves GET /challenge with response header Cf-Mitigated: challenge (header classification) embedding a same-fixture script at /cdn-cgi/challenge-platform/orchestrate.js (path classification), so BOTH relaxation points run: the document-CSP projection (sw.js addCSP) and the script-CSP projection + no-store skip (rewriteScriptResponse, challengeSubresourceSkip). GET /plain is the non-compat baseline document. - Drives the proxy via the REAL B5 opt-in UI (#challenge-compat checkbox + #url + click), not a synthesized arm header, in three isolated browser contexts (armed challenge / OFF challenge / OFF plain). The arm is birth-only so each share-open mints its own kernel tab. - The fixture serves NO Content-Security-Policy, so the kernel eval grant (targetDynamicCompileAllowed) is the SAME default-allow for every run; this holds allowDynamicCompile constant so the byte-identical comparison isolates the challenge projection as the only variable (honor-not-manufacture eval). Assertions (redacted trace: url-path-class, names-only cookies, status, through_zeroproxy bool; NEVER token/cookie/arm values, in records AND failure diagnostics): - (a) ARMED: the SW-synthesized document CSP reaching the page adds challenges.cloudflare.com to script/connect/frame/child; no wildcard egress; worker-src stays 'self' blob: (projection manufactures no eval). - (b) the internal X-ZP-Challenge-Compat marker is ABSENT from page-visible response headers (consumed-and-deleted at the SW layer). - (c) every armed challenge resource routes through the proxy (/zp/*); the challenge subresource is a through-proxy /zp/api/script; ZERO direct-egress requests (any non-through_zeroproxy request is a hard fail); the fixture sees only proxied-UA requests (browser never reaches the target directly). - (d) OFF: the challenge-document CSP is BYTE-IDENTICAL to the non-compat plain baseline, and ARMED equals OFF plus ONLY the additive challenge-host segments (stripping challenges.cloudflare.com from ARMED reproduces OFF byte-for-byte). scripts/test.mjs: the e2e and all modes now run node --test test/e2e/*.test.js (was hardcoded proxy.test.js), keeping proxy.test.js running too. Verified: npm run build + npm run test:e2e (2 pass: proxy + turnstile-compat); frozen invariants (Go internal/headers,zphttp,cookiejar 38; JS membrane-invariants,static-policy 27) green; npm run test:js 75; go test ./... 83; golangci native + GOOS=js GOARCH=wasm 0 issues; biome ci web scripts test exit 0. Op: extend
…on-guarantee Task B7: append an "Implemented: Increment 1 (challenge compatibility mode)" section to docs/cloudflare-turnstile/README.md documenting the shipped B1-B6 mechanism and its honest expectations. Docs only; no code changed. Records: - the opt-in (default OFF) and the single user surface (web/index.html challenge-compat checkbox on the openTarget path only; the cold handleShare share-link path stays unarmed by design); - the trusted window -> SW -> kernel arm hop (ZP_OPEN_SHARE -> createTab -> transportFetch authoritative X-Zp-Challenge-Compat-Arm set/delete -> kernel birth-only read), with the fetchThroughRuntime inbound strip, so a proxied page can never self-arm; - the two-signal gate (trusted arm AND header/URL classification) at every relaxation point; - exactly what is projected: challenge-host CSP allowances (no wildcard, no egress), eval honored-not-manufactured, and the no-store skip for classified SUBRESOURCES only (the challenge document stays no-store); - the internal X-ZP-Challenge-Compat marker consumed/deleted at the SW so it never leaks to the page; - no-egress and no-forgery guarantees; - HONEST expectations: this is COMPATIBILITY (stops the proxy breaking the legitimate human challenge), NOT a solver/forger/bypass/clearance guarantee; real-zone clearance is server-authoritative and verified via a human-run live smoke (ZP_TURNSTILE_LIVE convention, NOT wired into CI), while CI validates only the mechanism against a local fixture (B6) that never contacts Cloudflare; - the constituent commits B1-B6. Op: extend
…omplexity budget Split the over-budget functions in cmd/zeroproxy-server into small, single-responsibility units while preserving every byte of observable behavior (SOCKS5 wire protocol, asset allowlist + default-deny, CSP bytes): - readSOCKS5Connect (cc 19, cog 20) -> socks5Negotiate (greeting + method select + auth) / socks5SelectMethod (pure method pick) / socks5ReadRequest (command + address + port) / readSOCKS5Port. Write-then-check-0xff ordering and method preference (0x02 > 0x00) preserved exactly. - handle (cc 16) path-routing switch -> an ordered route table dispatched top-to-bottom, mirroring the switch 1:1 with the same default-deny fallthrough. - legacyZP nested switch -> flat map lookups (control redirects + asset allowlist) with the same /__zp/error/ prefix branch and default-deny. - zeroCSP/serviceWorkerCSP -> single cspWithScriptSrc(r, scriptSrc) helper, removing the fragile strings.Replace. Both byte-pinned script-src literals survive as call-site arguments (static-policy.test.js stays green). Add routing_test.go: table-driven httptest coverage of handle's dispatch, the asset allowlist, and default-deny (the routing refactor previously had no test net). No behavior change; restructuring only.
…t/policyFromRequest/adoptH2 under complexity budget
…exity budget Extract the shared expired-purge / match / sort loop into collectMatching, the read-side filter predicate into recordVisible, the comparator into sortByPathThenCreation, and the snapshot projection into snapshotFromRecord. cookies and VisibleRecords now share one predicate (VisibleRecords being the includeHTTPOnly=false, nil-ctx specialization of recordVisible), eliminating the duplicated loop+sort. Behavior preserved exactly: credentials=omit still short-circuits before the lock/purge; the HTTPOnly skip becomes the first predicate conjunct (same short-circuit, record still retained in jar); matched records still get LastAccessTime bumped (touch=true) and carried into the retained jar while VisibleRecords leaves it untouched (touch=false); both paths still compact expired records in place and write back j.records. cookies: gocognit 17->5, cyclop 16->6. VisibleRecords: gocognit 15->3, cyclop 14->4. All under cyclop<10 / gocognit<15 / nestif<4.
…statement_bindings under complexity budget
Parallel worktree workflows materialize .claude/worktrees/; keep it out of the tree so tooling scratch never gets committed. Op: compress
…r/stealth/network hooks under complexity budget Op: compress
… caller) NormalizeRelayServers had exactly one caller (shareFragment, same package) and same-package tests -- zero external consumers across the tree. Unexport it to shrink the package's public API surface (the original cleanup plan called for this). Compiler-verified rename; build + tests + golangci clean. Op: compress
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d194d2f10d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+263
to
+265
| if (!headers.has('X-ZP-Document-URL')) { | ||
| const entry = transportDocumentEntry(opt); | ||
| headers.set('X-ZP-Document-URL', entry && (entry.baseUrl || entry.targetUrl) || u); |
There was a problem hiding this comment.
When a proxied page bypasses the prelude and calls /zp/api/fetch?url=https://victim.example/... directly, the request still has an authorized client context but no trusted X-ZP-Document-URL; this branch defaults that document URL to the target URL (and also preserves any caller-supplied value). BuildHTTP1Request then feeds that value into the cookie jar's SameSite/credentials checks, so the target is treated as same-site and Strict/Lax/HttpOnly cookies can be attached to a cross-site subresource fetch whose response is readable by the page. Please set this metadata from the tab/entry document state and overwrite/delete page-provided values instead of accepting headers from the request.
Useful? React with 👍 / 👎.
…ver done) The /p/, /__zp/, /sw.js -> /zp/ compatibility redirects existed only to keep pre-cutover URLs working during the Phase-3 migration (PHASE3_PLAN.md:66 -- "may exist during migration only to redirect... not accepted steady-state surfaces"). No instance was deployed under the old scheme (confirmed by the operator), and no live code generates those spellings: the SW registers /zp/sw.js and shareurl emits /zp/p/. The redirects served no real consumers. Removed the 3 route entries, the redirectLegacyPage/redirectLegacySW/ redirectLegacy/legacyZP handlers, and the legacyControlRedirects/ legacyAssetNames allowlists. Legacy paths now fail closed (POLICY_BLOCKED 403) through the existing handle() default-deny -- a STRONGER posture than the prior redirect, with the security boundary fully preserved. routing_test now pins the new contract (legacy paths denied, not redirected). Build + go test ./... + golangci native+wasm all clean; no orphaned symbols, imports, or generators. BREAKING CHANGE: /p/*, /__zp/*, and root /sw.js now return 403 instead of a 307 redirect to the canonical /zp/ path. Op: compress
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0104c803a0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+165
to
+166
| if tab.CookieJar != nil && req.Header.Get("X-Zp-Fetch-Credentials") != "omit" { | ||
| tab.CookieJar.SetCookies(finalURL, resp.Cookies()) |
There was a problem hiding this comment.
When a runtime fetch() uses the default credentials: "same-origin" to a cross-origin target, this condition still stores any Set-Cookie from the final response because it only excludes omit. The redirect-hop path already uses policyAllowsCookies, so the final response can diverge: a cross-origin response that the browser would ignore can poison the tab jar and be sent later on navigations or include requests to that origin. Please gate final response cookie capture with the same request policy/origin check used in redirect handling, not just != "omit".
Useful? React with 👍 / 👎.
Comment on lines
+1957
to
+1960
| const resp = await fetchThroughRuntime(target.href, { method, body, headers: reqHeaders }); | ||
| const html = await resp.text(); | ||
| virtualURL = new URL(target.href); | ||
| baseURL = virtualURL.href; |
There was a problem hiding this comment.
For a non-GET form submission whose target redirects, fetchThroughRuntime exposes the transport's final URL via resp.url, but this code resets the document state to the original form action. Because the new HTML is written into the existing window and the runtime prelude is already installed, the boot config embedded in the transformed response cannot correct virtualURL/baseURL; subsequent relative links, referrers, and document.URL remain stuck on the pre-redirect action URL. Use the response URL (when present) before updating the virtual document state and share route.
Useful? React with 👍 / 👎.
…ant challenge kernel logic
…e tests accordingly
…ment Rust-based import map rewriting
…implement worker global scope property masking
…rame behavior test coverage
…RL rewriting context with tab and runtime token tracking
…ntime facades for DOM, network, and performance shielding
…ualize frame property access and messaging sources
…nalysis for e2e tests
…ngAccessor and improving document URL/origin virtualization.
… and repository documentation
…o 24.16.0 in CI workflows
…Linux persona to expected deltas
Op: correct Restores: CI Biome lint
… improved Request facade handling
…n handling for about:srcdoc frames
rabbitson87
added a commit
that referenced
this pull request
Jun 5, 2026
…cause) PHASE2_STATUS.md: * Cumulative acceptance: cargo 86→93 / static-policy 29 / build clean * C1 row flipped [~] → [x] with the Rust ws_client landing * D2 row flipped [~] → [x] with the sourcemap composer landing * Follow-up #1 (WS transport) marked landed with full RFC 6455 invariant list * Follow-up #2 (sourcemap composition) marked landed with composer note * Follow-up #7 (patch-mode) updated with SW wire-up + applier coverage * Acceptance signal updated for the cookie-jar test repoint + new tests PRODUCTION_ROLLOUT.md: * C1 carry-over entry struck through with the ws_client landing * D2 carry-over entry struck through with the sourcemap composer landing * Patch-mode wire-up carry-over entry struck through .ai/trap-notebook/INDEX.md, real-site-compat.md: * 2026-06-05 NAVER entry — full diagnosis (Object.getOwnPropertyNames enumeration), fix (window.ZP / window.ZeroProxyRT closure-capture + delete, AST loop cap), and the WebView2 + Tauri vs general chromium distinction. Edge real-browser verification confirms NAVER full page render with the fingerprint hide. * WebView2 (Tauri host) vs chrome.exe / msedge.exe distinction clarified to prevent future mis-attribution to "Chrome" of the WebView2 wedge * Pre-existing trap-notebook subpages (build-deploy / rewriter / sw-integration / tls-fingerprint / transport-regression / wasm-page-rt / README) added to repo * .ai/design/landing/ + .ai/zp-page-rt-*.md (earlier session drafts) added to repo 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
rabbitson87
added a commit
that referenced
this pull request
Jun 8, 2026
Two threads land together: #1 Embedded TURN (D5 polish completion) New internal/rtcgw/turn.go (~210 LOC) brings pion/turn/v4 in-process. Auth via RFC 7635 long-term TURN-REST short-term creds — each IssueICEServerCreds("") call returns a fresh (urls, username=<expiry-unix>:<label>, credential=HMAC-SHA1(secret, username)) tuple, valid for the configured TTL (default 30 min). main.go gains -rtc-turn-addr / -rtc-turn-public-addr / -rtc-turn-external-ip / -rtc-turn-realm / -rtc-turn-secret. Server struct carries the rtcTURN handle; serveConfig issues a fresh tuple per request and emits it as `"rtcICEServers":[...]` alongside the existing wtGateway/rtcGateway fields. SW refreshRuntimeConfig now parses rtcICEServers; buildRuntimePrelude threads it into the boot JSON. runtime-prelude's ZPRTCPC swaps the force-empty `safeConfig.iceServers = []` for `= issuedICEServers`, sourcing the array from `boot.rtcICEServers` (falls back to `[]`). Defense-in-depth posture stays: page-supplied iceServers is still never used; only operator-issued embedded TURN creds reach native. Tests: TestTURNServerLifecycle (startup + cred issuance + Close + label-based username uniqueness) + TestTURNServerRejectsBadAddr. Static-policy +1 (D5 embedded TURN: pion/turn server + short-term creds + page-realm iceServers wiring). The existing D5 client test updated to pin `safeConfig.iceServers = issuedICEServers` plus the `boot.rtcICEServers` source path. #2 MDN diagnosis corrected — NOT a transport hang The 2026-06-08 trap entry hypothesized "MDN transport stall" based on harness timeout. Re-reading developer.mozilla.org-errors.json captured during the failed run shows two `403 (Forbidden)` console errors — Cloudflare anti-bot rejected our request immediately. The "hang" was just the page-realm being unresponsive AFTER the SW returned the error page; the title-poll loop hit puppeteer protocolTimeout on a renderer that had nothing to update. This is the same class of failure as gosuda.org Cloudflare turnstile, NOT a transport bug. Trap notebook updated, the real-site-regression harness MDN comment corrected, PHASE2_STATUS E4 note updated. Phase 3 fingerprint hardening (Chrome 148 spec → latest, h2 frame ordering audit, request header set/case parity, real ECH negotiation) is the path forward. Tests: cargo test --workspace: 152/0 pass go test ./internal/rtcgw: 8/8 (2 new TURN tests) go test ./...: all internal packages green node test/js/static-policy.test.js: 48 → 49 pass node scripts/dogfood-matrix.mjs: 3/3 in 7.2 s (post-TURN) zp_page_bundle_bg.wasm: 375.6 KB (still ≤ 500 KB E3 target) PHASE2_STATUS.md D5 row: Phase 3 polish carry-over reduced to "per-tab session attribution gated on X-ZP-Runtime-Token" only. PRODUCTION_ROLLOUT.md D5 carry-over rewritten to reflect the closure. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
rabbitson87
added a commit
that referenced
this pull request
Jun 12, 2026
#1 NAVER hydration regression — dynamic import URL rewrite NAVER's `gfp-core.js` does `import('./gfp-display-glog-logger.js')` from `ssl.pstatic.net`; the page realm's base is `proxy.localhost`, so the relative path resolved against the WRONG origin and 404'd at `/zp/api/gfp-display-glog-logger.js`. Hydration broke → search box only. The "regression vs the past" was actually `e33e4cb` (drop dead initRewriter call) — before that, every external script was block-stubbed and NAVER showed only its static HTML (which looked "correct" to users). Fix: zp-rewriter's `RewriteVisitor` gains `visit_import_expression`. String-literal sources are resolved against `RewriteOpts.target_url` and patched to `/zp/api/script?u=<percent-encoded-abs>&kind=module` so the SW's existing `/zp/api/script` route fetches + rewrites the module body through the proxy. Computed-source dynamic imports (variables, templates) stay as-written; inner global identifiers still go through the membrane. URL crate avoided: `url 2.5` would pull ~250 KB of ICU IDN handling into the page bundle (E3 size guard ≤ 500 KB violated). Replaced with a hand-rolled RFC 3986 §5.3 minimal resolver (`resolve_module_base`) — ASCII URLs only, 4-branch dispatch (`./` / `../` / `/abs` / `https://...`) + segment collapse. Page bundle 850→381 KB after the fix (still well under 500 KB). Tests: - 5 cargo (`dynamic_import_*` covering relative / abs https / data: skip / bare specifier skip / computed-source membrane) - static-policy `NAVER dynamic import fix: rewriter routes literal import() through /zp/api/script` #2 Perf telemetry — rewrite cache hit ratio + latencies New `rewriteStats = {invocations, hits, misses, rewriteLatencyMs, cacheKeyLatencyMs}` counters in sw.js. `rewriteScriptResponse` wall-clock-times the SHA-256 cache key compute AND the Rust `ZPBundle.rewriteScript` call so future tuning has data instead of guesses. `__zpKernelProbe` response gains `rewriteStats` with hitRatio + cacheKeyShare derived metrics + raw counters. `cacheKeyShare > 50%` flags the SHA-256 key as a pessimization (small LRU + mostly-miss cold path that still pays the digest cost every time). Static-policy `perf telemetry: SW exposes rewrite cache hit/miss + latency counters` pins the counters so a refactor can't silently drop them. Diagnostic helper: scripts/probe-naver.mjs (puppeteer harness that loads launcher → submits target → captures kernel rust trace before and after the doc fetch + dumps console errors / page state). Known follow-up (separate Phase 3 track): NAVER's `gfp-display-glog-logger.js` fetch now routes through `/zp/api/script?u=...` correctly but the upstream fetch returns 403 (anti-bot rejection). Direct `curl` to the same URL is 200 OK — gap is in our request shape (headers / cookies / sec-fetch-* metadata). Captured in .ai/trap-notebook/INDEX.md 2026-06-09 rewriter entry §6 with the NAVER probe artifact for repro. Tests: cargo test --workspace: 152/0 (+5 dynamic_import) go test ./...: all internal packages green node test/js/static-policy.test.js: 49 → 51 pass zp_page_bundle_bg.wasm: 850 KB → 381 KB (-55%, ≤ 500 KB target) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
rabbitson87
added a commit
that referenced
this pull request
Jun 12, 2026
…der debug log #1 Root cause transportFetch's header build order let the page-side User-Agent win over the canonical Chrome 148 UA. In a real-Chrome WebView2 that's harmless (the page UA already is the genuine browser UA), but in any automation context (puppeteer headless, some Edge WebView2 modes) the page-side UA contains the literal substring `HeadlessChrome` — NAVER WAF instantly 403s every request whose User-Agent contains that substring, killing all ad SDK fetches (gfp-display-sdk.js, gfp-display-glog-logger.js, …). The leak was through the first `headers.entries()` loop: `new Headers(opt.request.headers)` preserves UA, the loop adds it to seen/headerEntries, and the later `pushOnce('user-agent', ZP.TARGET_USER_AGENT)` no-ops because seen already contains 'user-agent'. Fix: move `pushOnce('user-agent', ZP.TARGET_USER_AGENT)` BEFORE the entries() loop. Canonical Chrome 148 UA wins regardless of the page-realm supplied value. Real-Chrome operators still get the same Chrome 148 UA — no functional change for them. #2 Header debug ring buffer New `outgoingHeaderLog` (MAX_HEADER_LOG = 16) captures the last N outgoing transportFetch header arrays. `__zpKernelProbe` response gains `outgoingHeaders` so probe-naver.mjs (and any future anti-bot debug) can dump exactly what reached the WAF. This is what found the HeadlessChrome UA in the first place — no codebase grep would have surfaced it because the leak was through default Headers constructor behavior, not an explicit set. Future trap-notebook entries for anti-bot debug should start by dumping outgoingHeaders before forming hypotheses. #3 probe-naver.mjs upgrade Now dumps rustTrace + rewriteStats + outgoingHeaders in both the baseline (launcher-loaded) and post-NAVER-fetch (fresh launcher tab) probes. Per-fetch header lists are printed verbatim so a side-by-side with real-Chrome wireshark capture spots gaps fast. Verification (probe-naver.mjs https://www.naver.com/): Before: 28 console errors, NAVER body 95 bytes (search box only) After: 2 console errors (1 unrelated CSP warning + 1 deeper ad SDK 403), NAVER main 200 OK, body 260,345 bytes ✓ The remaining 1 deeper-chain 403 is documented as Phase 3 work — probably cookie/origin-specific to gfp-display-glog-logger.js. The header debug log makes that next debug step a single probe run. Tests: cargo test --workspace: 152/0 (no change) go test ./...: all internal packages green (no change) node test/js/static-policy.test.js: 51 → 52 pass zp_page_bundle_bg.wasm: 381 KB (unchanged, E3 guard intact) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces comprehensive linting and code quality enforcement for Go, Rust, and JavaScript, as well as new developer documentation outlining architecture, complexity gates, and workflow conventions. The main changes include adding a strict
golangci-lintconfiguration, a new CI job for linting across languages, and extensive documentation to guide contributors on project boundaries, complexity management, and best practices.Linting and CI improvements:
lintjob to.github/workflows/ci.ymlthat runs Go, Rust, and JavaScript linters and formatters in CI, includinggolangci-lint,cargo clippy, and Biome, with strict complexity gates enforced for all languages.cache-dependency-path: package-lock.json.Go linting configuration:
.golangci.ymlfile with a strict configuration: all complexity gates (cyclop,gocognit,nestif) are enforced as hard errors, specific linters are enabled, and detailed exclusions are documented for legacy or intrinsic findings. Test files and generated code are appropriately excluded.Documentation and developer guidance:
AGENTS.md, a comprehensive developer guide covering architecture boundaries, security invariants, complexity/linting policies, and build/test workflow traps. This document is required reading for contributors working on membrane or protocol code.AGENTS.mdfromCLAUDE.mdto ensure visibility in agent-related documentation.README.mdto reflect the current acceptance-grade implementation phase.