You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit a1c4151
Browse filesBrowse the repository at this point in the historyBrowse files
Get Bots past bot checks: headed browsers, Web Bot Auth, a residential proxy option, Bun 1.4.3, and a handoff that keeps its page (CopilotKit#772)
* Keep the challenge page through a computer restart
Opening a Cloudflare-fronted page could restart the computer, and the
handoff that followed showed the person about:blank with nothing to solve.
The restart: the egress filter pinned each connection to its checked
addresses with autoSelectFamily, and Bun's multi-address connect can throw
from a timer after the attempt is gone (oven-sh/bun#25633). Nothing catches
that throw, so the process exited. Cloudflare hosts resolve to several
addresses and a challenge page opens and drops many connections, which is
when it fired. The filter now tries one checked address at a time, IPv4
first, with its own per-attempt timeout, and the process turns
multi-address connect off for every other connect.
The blank page: a restarted or relaunched browser comes back on
about:blank and nothing sent it anywhere. A handoff request now records
the page it was raised on, and a request made from a blank browser shortly
after a restart inherits the interrupted request's page. When a request is
made or taken and the browser is blank, the computer opens that page again
through the normal navigation path, so egress and page-access rules still
apply. Spent __cf_chl_* challenge tokens are not kept.
* Run on Bun 1.4.3, where the multi-address connect crash is fixed
Bun 1.4.0 fixed the autoSelectFamily timer that threw
'null is not an object (evaluating context)' and exited the computer
(oven-sh/bun#25633, fixed by oven-sh/bun#32660). Bun's own regression
fixture fails on 1.3.14 and passes on 1.4.3. Every pin moves: the root
packageManager, CI and release workflows, the computer, server,
supervisor and Bot images (computer images by digest), the desktop
runtime and its checksums, and prompt.txt. The single-address egress
connect stays as defence in depth.
* Run hosted computers' browsers headed by default
Headless Chromium is what bot checks catch first: it says HeadlessChrome
and Cloudflare challenges it on sight, even from a residential address.
Every hosted computer runs on Linux (a container, a pod, the desktop
app's engine) and already ships Xvfb, so the managed browser now defaults
to headed there, drawing to the computer's own display behind the same
live screen. Compose defaults the shared computer and supervised ones to
headed too. A computer run directly on macOS or Windows stays headless,
where headed would open windows on the desktop, and
COMPUTER_BROWSER_MODE=headless still chooses headless anywhere.
* Sign every request a Bot's browser sends with Web Bot Auth
Cloudflare lets a signed agent through where it would challenge an
unknown browser, which is how OpenAI's agent gets past it. The server now
holds one Ed25519 key, derived from KEY_ENCRYPTION_KEY or given as
WEB_BOT_AUTH_KEY, and publishes its public half, signed over
("@authority";req), at /.well-known/http-message-signatures-directory.
The key reaches each computer inside the network-policy push the server
already sends.
The computer signs in one browser-level DevTools Fetch session, so every
request is covered whatever sent it: pages, redirects, cross-site frames
in their own processes, dedicated workers, service workers and their
script fetches. Headers are added as the request leaves the browser, so
cookies, CORS, TLS and the egress filter are unchanged. Signature-Agent
is the bare string Cloudflare documents, with @authority and
signature-agent covered, a 60 second expiry and a nonce.
WEB_BOT_AUTH unset means on when the public URL is https on a public
address, which a verifier can fetch the key from, and off otherwise. A
headless browser also stops calling itself HeadlessChrome, on the wire
and to scripts.
* Chart settings for headed browsers, Web Bot Auth and a residential proxy
computers.browserMode (headed by default) and computers.egressProxy reach
the shared computer and every per-Bot sandbox through one helper, so the
two modes cannot drift. The residential or ISP proxy URL is a secret
(secrets.egressProxyUrl or a Secret of your own) and arrives as
EGRESS_PROXY_DEFAULT behind the egress filter. It is off by default, and
the chart refuses it with network policies on and no path to the proxy,
or switched on with no URL. config.webBotAuth and secrets.webBotAuthKey
pass Web Bot Auth through. Every new key is guarded for
--reuse-values upgrades.
The proxy is tested end to end against a gateway that demands Basic
credentials, for plain HTTP and CONNECT, including that the policy is
decided before the gateway is asked.
* Document getting past bot checks
How a Bot's browser presents itself and what each setting changes:
headed Chromium, Web Bot Auth with the one-time Cloudflare registration,
and a residential proxy as the operator's choice. The handoff is
described as the designed fallback for a check a Bot cannot pass.
* Get the Bun 1.4 and headed-browser CI runs passing
The tests job never installed agent-computer's own dependencies, so the
Web Bot Auth tests could not import web-bot-auth under the root run. It
now installs them, as it already does for the other packages that keep
their own lockfile.
Bun 1.4 reports a refused fetch as a TypeError with code ECONNREFUSED
instead of an Error with code ConnectionRefused, so the tool-selection
driver's stopped-listener check matched neither shape. It now matches on
the error code from either version.
The headed-browser test navigated to a data: URL, which /navigate has
refused since it was hardened; it had never run in CI until this branch
asked for it. It now serves the page from a local http server.
* Keep a handoff's page only across a restart, and reopen it in the background
A stop or reset left the request interrupted exactly like a process restart, so a help request
from a blank browser within ten minutes reopened the page the reset was meant to forget. Only a
restart underneath the request (the process or the browser coming back up) now marks it as
keeping its page; the stop and reset routes clear it, including after the browser close that
reaches renewRun first.
A finished request no longer keeps its page. The address can carry a one-time token and the
record stays on disk for a month, so finish() drops it for every outcome but a restart, a
restart's page is dropped once the help TTL passes, and records written before this are cleaned
when they are read.
The help call and Take control no longer wait for the page to reopen, which could take a cold
browser start plus a full navigation timeout. The restore runs in the background, one at a time
per Bot, and gives up if a person has sent input or the request has ended by the time the
browser is ready.
0 commit comments