Skip to content

Commit c1ff100

Browse files
committed
docs(readme): v0.6 — batch flagship + dedup ledger + ARIA RLE + footnoting + CDP resilience
- Tool count 64 → 65 (everywhere: intro, in-numbers, install verify, tool tree heading, comparison matrix, architecture box). - batch ⭐ added to tool tree as flagship category + new highlight section with a 5-call HN example and stop_on_error note. - 🔬 token-efficiency expanded with three v0.5 mechanisms that were shipped but undocumented: • cross-call dedup ledger ({_unchanged_since, _hash}) w/ before- after example • ARIA pattern grouping (lossless RLE for repeating snapshots, ~70% smaller on HN comments) • URL footnoting (host dedup in extract_links via _hosts table, ~50% smaller on link-heavy pages) - 🩹 New CDP schema resilience section explaining the three nodriver monkey-patches and their idempotence + partial-failure guarantees. - in-numbers row added: "CDP schema drift — resilient". - TODO (features) gains two playwright-parity items: • full request interception graph (Fetch.requestPaused + match DSL) • battle-test ARIA tree on real-world edges (regression corpus)
1 parent b038b67 commit c1ff100

1 file changed

Lines changed: 81 additions & 9 deletions

File tree

‎README.md‎

Lines changed: 81 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
# umbra
66

77
> **The de-facto MCP server for stealth browser automation.**
8-
> Real Chrome, 0% creepjs detection, 31/31 sannysoft, 64 broad tools, multi-browser orchestration, encrypted sessions, prompt-injection signaling, and live human handoff over a Cloudflare tunnel — for AI agents that need to browse the web like a human, not a bot.
8+
> Real Chrome, 0% creepjs detection, 31/31 sannysoft, 65 broad tools, multi-browser orchestration, encrypted sessions, prompt-injection signaling, and live human handoff over a Cloudflare tunnel — for AI agents that need to browse the web like a human, not a bot.
99
1010
> *umbra — the darkest part of a shadow, where light is fully blocked.*
1111
@@ -15,7 +15,7 @@ Built by merging the best parts of [obscura](https://github.com/h4ckf0r0day/obsc
1515

1616
## 🪙 token efficiency
1717

18-
Counter-intuitively, umbra costs LESS context than minimal browser-MCPs (incl. playwright-mcp) on any real agent session — its 64-tool catalog adds ~13KB upfront, but per-call savings recover that within 3 calls and dominate after that.
18+
Counter-intuitively, umbra costs LESS context than minimal browser-MCPs (incl. playwright-mcp) on any real agent session — its 65-tool catalog adds ~13KB upfront, but per-call savings recover that within 3 calls and dominate after that.
1919

2020
| | upfront catalog | typical 5-call session | 20-call session |
2121
|---|---|---|---|
@@ -41,11 +41,12 @@ Toggle off via `set_verbosity('full')` when you need raw byte-exact output.
4141
| **creepjs `headless`** | **0 %** (matches vanilla Chrome) |
4242
| **creepjs `stealth`** | **0 %** |
4343
| **CDP automation tells** stripped | `webdriver` `cdc_*` `$cdc_` `_phantom` `_selenium` `__webdriver_*` `__nightmare` ... |
44-
| **MCP tools** | **64** (broad primitives, not 95 narrow ones) |
44+
| **MCP tools** | **65** (broad primitives + `batch` flagship, not 95 narrow ones) |
4545
| **headless** | real GPU via `--headless=new + ANGLE Vulkan` (no SwiftShader tell) |
4646
| **TLS / JA3** | real Chrome stack + optional `curl_cffi` for raw HTTP |
4747
| **WebRTC** | mDNS-aware SDP filter (real-Chrome behavior, no LAN IP leak) |
4848
| **handoff** | live remote-view via `cloudflared` Quick Tunnel (works VPS → home laptop) |
49+
| **CDP schema drift** | resilient — survives Chrome field churn (e.g. dropped `sameParty`) without hangs |
4950

5051
---
5152

@@ -103,7 +104,7 @@ claude mcp add-json umbra '{
103104
}'
104105
```
105106

106-
Then restart Claude Code → `/mcp` should show `umbra` with 64 tools.
107+
Then restart Claude Code → `/mcp` should show `umbra` with 65 tools.
107108

108109
For Cursor / Claude Desktop / other MCP clients, edit their `mcp_servers` config with the same shape.
109110

@@ -124,6 +125,10 @@ pytest -m e2e -v -s # full regression suite (boots real Chrome, ~60s)
124125

125126
- [ ] **Proxy pool rotation** — currently `StealthOptions(proxy="...")` accepts one proxy per session. For high-volume scraping or geo-distributed scraping, add a `proxy_pool=[...]` option that round-robins (or rotates per-tab / per-N-requests / on-403). Pair w/ residential providers (smartproxy, iproyale, brightdata) for IP reputation. ~80 LoC + a per-tab proxy override via CDP `Network.setExtraHTTPHeaders` + `--proxy-server` per browser instance.
126127

128+
- [ ] **Full request interception graph** — current `block_urls` + `dynamic_hook` cover block/header-injection patterns, but playwright-mcp wins on deep interception: per-request `route()` w/ `fulfill / continue / abort`, body rewrite, response stubbing, HAR replay, conditional-on-headers matching. Build via CDP `Fetch.enable + Fetch.requestPaused` (already wired for hooks) + a richer match DSL — `route(pattern, handler)` returning `{action, status, body, headers, delay_ms}`. Unlocks offline replay + auth-token swap + chaos testing.
129+
130+
- [ ] **Battle-test the ARIA tree on edge cases** — fantoma-derived snapshot covers the 95% case (forms, lists, dialogs, nav) but real-world weirdness still exposes gaps: shadow-DOM-inside-iframe-inside-shadow-DOM, custom elements w/ delegated focus, `<canvas>`-rendered "trees" (Figma/Notion), virtual-scroll lists where ARIA indexes shift mid-snapshot, `aria-owns` cross-references, RTL/i18n role inflections. Need a regression corpus (gmail, github, notion, figma, linear, jira, gov forms) + property-based tests so we don't regress as nodriver/Chrome update. Playwright's accessibility tree has a decade of these baked in — ours is ~6 months.
131+
127132
---
128133

129134
## 🤖 use as an MCP server (the main use case)
@@ -137,11 +142,11 @@ claude mcp add-json umbra '{
137142
}'
138143
```
139144

140-
Now your agent has 64 MCP tools for stealth Chrome automation. Cursor, Claude Desktop, Claude Code — anything MCP.
145+
Now your agent has 65 MCP tools for stealth Chrome automation. Cursor, Claude Desktop, Claude Code — anything MCP.
141146

142147
---
143148

144-
## 🧰 the 64 tools
149+
## 🧰 the 65 tools
145150

146151
```
147152
┌─ browser spawn / close / list_browsers / close_browser /
@@ -179,13 +184,33 @@ Now your agent has 64 MCP tools for stealth Chrome automation. Cursor, Claude De
179184
│
180185
├─ files upload_file / setup_downloads / wait_for_download
181186
│
182-
└─ TLS tls_fetch (raw HTTP w/ Chrome JA3+JA4)
187+
├─ TLS tls_fetch (raw HTTP w/ Chrome JA3+JA4)
188+
│
189+
└─ batch ⭐ flagship batch (N tools in one round-trip; composes w/ dedup)
183190
```
184191

185192
---
186193

187194
## 🎯 highlight tools
188195

196+
### `batch` ⭐ flagship — N tools in one MCP round-trip
197+
198+
```python
199+
batch([
200+
{"tool": "navigate", "args": {"tab_id": "t0", "url": "https://news.ycombinator.com"}},
201+
{"tool": "wait_for_text", "args": {"tab_id": "t0", "text": "Hacker News"}},
202+
{"tool": "aria_snapshot", "args": {"tab_id": "t0"}},
203+
{"tool": "extract_links", "args": {"tab_id": "t0", "limit": 30}},
204+
{"tool": "extract_markdown","args": {"tab_id": "t0"}},
205+
])
206+
# → {"results":[...5 entries with ok/data/ms each...],
207+
# "elapsed_ms":1840, "ok_count":5, "fail_count":0}
208+
```
209+
210+
Serial in declared order, single MCP round-trip. Saves protocol framing per call AND composes with cross-call dedup (identical re-calls inside the batch return `_unchanged_since` instead of full payloads). Use it whenever you have ≥2 calls in mind — it's almost always the right choice.
211+
212+
`stop_on_error=True` short-circuits the batch on first failure (default: keep going + report fail_count).
213+
189214
### `handoff_start` — when you hit a captcha, hand the wheel back
190215

191216
```
@@ -268,7 +293,7 @@ list_browsers()
268293
| tracker/fp-script blocking | ✓ (3520) | ✗ | ✗ | ✓ (3520 + dynamic hooks) |
269294
| session warming (cookie age) | ✗ | ✗ | ✗ | ✓ (4 profiles) |
270295
| live human handoff | ✗ | ✗ | ✗ | ✓ (cloudflared tunnel) |
271-
| MCP tool surface | ✗ | ✗ | ✓ (95 narrow) | ✓ (64 broad) |
296+
| MCP tool surface | ✗ | ✗ | ✓ (95 narrow) | ✓ (65 broad) |
272297
| prompt-injection signaling | ✗ | ✗ | ✗ | ✓ (`_untrusted: true` on all extraction) |
273298

274299
---
@@ -319,13 +344,60 @@ every MCP tool response goes through `_compact()`:
319344

320345
**24% average wire-byte savings** on real-world pages (test data on HN/wikipedia). flip with `set_verbosity('full')` when you need raw.
321346

347+
### cross-call dedup ledger
348+
349+
Identical repeat calls return `{"_unchanged_since": "cN", "_hash": "..."}` instead of the full payload — the data is unchanged from call cN, so the agent reuses what it already has in context. Pass `force_refresh=True` to bypass.
350+
351+
```python
352+
extract_text('t0') # → call c5: full {text:"...",length:8421,...}
353+
extract_text('t0') # → call c6: {"_unchanged_since":"c5","_hash":"a7f2..."} ← saved 8KB
354+
```
355+
356+
Pairs perfectly with `batch` — you can blast `[snapshot, snapshot, snapshot]` after each interaction; only the deltas come back.
357+
358+
### ARIA pattern grouping (RLE for snapshots)
359+
360+
Long lists (HN comments, search results, file trees) with repeating `(role, name)` cycles get run-length-encoded losslessly:
361+
362+
```
363+
[12-77] cycle×13 (period 5): link("Comments"), link("Permalink"), link("Save"), button("Vote"), text("user")
364+
↳ 66 lines collapsed into 1 — agent still knows the exact range and what's in each cycle
365+
```
366+
367+
Detects period 1–6 with ≥2 reps. Real HN comments page: ~70% smaller snapshot.
368+
369+
### URL footnoting (host dedup in `extract_links`)
370+
371+
Repeated hosts get factored out once:
372+
373+
```
374+
{"_hosts": {"h1":"https://github.com", "h2":"https://news.ycombinator.com"},
375+
"links": [["h1","/user/foo"], ["h1","/issues/123"], ["h2","/item?id=456"], ...]}
376+
```
377+
378+
~50% smaller on link-heavy pages. Reconstruct via `_hosts[h] + path`.
379+
380+
---
381+
382+
## 🩹 CDP schema resilience
383+
384+
nodriver's CDP parser hardcodes Chrome protocol field names — when Chrome changes the schema between releases, the parser KeyErrors. Worse, the listener task dies on the unhandled raise → every subsequent CDP call on that tab hangs forever (no awaiter ever wakes up).
385+
386+
umbra ships three monkey-patches in `umbra/nodriver_patch.py` to make this class of bug impossible:
387+
388+
1. **`Transaction.__call__`** — every parser exception becomes `future.set_exception(...)` so the awaiter gets a real error, never a hang.
389+
2. **`Connection._listener`** — wraps the per-message dispatch so a single bad parse can't kill the listener task; future calls keep working.
390+
3. **`Cookie.from_json`** — tolerant of Chrome 146+ dropping `sameParty` (matches the pattern already used in `CookieParam.from_json`; upstream inconsistency).
391+
392+
Patches are **idempotent** (per-class flag + module-level short-circuit, safe to call N times) and **partial-failure tolerant** (each patch runs in its own try/except — one failing doesn't block the others). Applied automatically at `umbra.browser` import — zero config.
393+
322394
---
323395

324396
## 🏗️ architecture
325397

326398
```
327399
┌────────────────────────────────────────────┐
328-
│ FastMCP server (umbra.server, 64 tools) │
400+
│ FastMCP server (umbra.server, 65 tools) │
329401
│ + _compact() minification │
330402
│ + _untrusted prompt-injection signaling │
331403
└────────────────────────────────────────────┘

0 commit comments

Comments
 (0)