Skip to content

Commit b7050b1

Browse files
committed
Auto-merge upstream openclaw/openclaw
2 parents 2ef6be2 + 3254b7f commit b7050b1

35 files changed

Lines changed: 1250 additions & 341 deletions

extensions/browser/skills/browser-automation/SKILL.md

Lines changed: 38 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -48,40 +48,56 @@ Use this skill when you need the `browser` tool for anything beyond a single pag
4848

4949
## Code Mode Loop
5050

51-
When `tools.codeMode` is enabled, call the Browser tool from exec cells:
51+
When `tools.codeMode` is enabled, the Browser tool has no normal turn — it is cataloged behind `exec`/`wait`. Call it from exec cells as an async global, using the callable name the exec quick index advertises for the Browser tool (normally `browser`; colliding names get suffixed, and a client tool can win an identical name). An exact `catalog.search("browser")` returns a handle already bound to the effective callable name, so resolve the handle in each cell and call it instead of hard-coding the literal global; an empty result means the Browser tool is not cataloged in this run.
5252

53-
```javascript
54-
const browserTool = "openclaw:browser:browser";
55-
let previousSnapshot = "";
56-
const callBrowser = async (input) => await tools.call(browserTool, input);
57-
```
58-
59-
Keep the same labeled tab through the loop, and alternate reads with actions:
53+
Keep the same labeled tab through the loop, and alternate reads with actions. Each `exec` cell starts a fresh VM — bindings from a completed cell are gone in the next, and only runs left `waiting` keep their state until `wait` resumes them — so carry comparison state across cells by returning it and re-embedding the returned values in the next cell:
6054

6155
```javascript
62-
const snapshotCall = await callBrowser({
56+
// previous = the url/newElements returned by the last completed cell (a fresh
57+
// VM runs this cell, so prior bindings do not exist here).
58+
const previous = { url: "https://example.com/inbox", newElements: 0 };
59+
const [browser] = await catalog.search("browser", { limit: 1 });
60+
const details = await browser({
6361
action: "snapshot",
62+
snapshotFormat: "ai",
6463
targetId: "task",
6564
refs: "aria",
6665
interactive: true,
6766
});
68-
const details = snapshotCall?.result?.details ?? {};
69-
const snapshot = (snapshotCall?.result?.content ?? []).map((block) => block?.text ?? "").join("\n");
70-
const relevant = snapshot
71-
.split("\n")
72-
.filter((line) => /submit|dialog|error|\[new\]/i.test(line))
73-
.slice(0, 12);
74-
const changed = snapshot !== previousSnapshot;
75-
previousSnapshot = snapshot;
76-
return { targetId: details.targetId, url: details.url, relevant, changed };
67+
const changed =
68+
details?.url !== previous.url ||
69+
(details?.newElements ?? 0) > 0 ||
70+
details?.blockedByDialog === true;
71+
return {
72+
targetId: details?.targetId,
73+
url: details?.url,
74+
newElements: details?.newElements,
75+
stats: details?.stats,
76+
changed,
77+
};
78+
```
79+
80+
- Code-mode calls return the tool's structured `details` directly (`targetId`, `url`, `newElements`, `stats`, `blockedByDialog`); rendered page text is not returned to code cells.
81+
- To read text inside code mode, run a targeted `act` evaluate (requires the evaluate capability; `browser.evaluateEnabled` can disable it) and keep the returned value bounded, because page-script output is untrusted:
82+
83+
```javascript
84+
const [browser] = await catalog.search("browser", { limit: 1 });
85+
const read = await browser({
86+
action: "act",
87+
kind: "evaluate",
88+
fn: "() => document.body.innerText.slice(0, 2000)",
89+
targetId: "task",
90+
});
91+
return { url: read?.url, text: read?.result };
7792
```
7893
79-
- Request interactive-only snapshots and filter them in code before returning.
80-
- Return only the handful of relevant elements; never return the full tree.
81-
- Keep `previousSnapshot` between cells when a local diff helps explain a change.
94+
When evaluate is unavailable, keep the loop on structured state only.
95+
96+
- Return only the fields the next step needs; never return the whole details object.
97+
- Completed cells share no state: re-embed the previous cell's returned `url`/`newElements` in the next cell, or keep the comparison inside one cell. Only `waiting` runs persist, resumed by `wait`.
8298
- Interleave each act with a URL or tabs check before the next dependent act.
8399
- If a batch returns `aborted`, take a fresh snapshot before continuing.
84-
- If `[new]` markers appear, inspect those elements first, then update the saved snapshot.
100+
- If `newElements` is positive, inspect those elements first, then update the re-embedded state.
85101
- Use separate act calls when navigation is expected between steps.
86102
87103
## Tab Hygiene

extensions/browser/src/browser/cdp.test.ts

Lines changed: 43 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -354,10 +354,15 @@ describe("cdp", () => {
354354
).rejects.toBe(reason);
355355
});
356356

357-
it("does not create a target when cancellation arrives during endpoint discovery", async () => {
357+
it("cancels hanging endpoint discovery without creating a target", async () => {
358358
const controller = new AbortController();
359359
const reason = new Error("cancel during endpoint discovery");
360360
const methods: string[] = [];
361+
let releaseDiscovery: (() => void) | undefined;
362+
let markDiscoveryStarted: (() => void) | undefined;
363+
const discoveryStarted = new Promise<void>((resolve) => {
364+
markDiscoveryStarted = resolve;
365+
});
361366
const wsPort = await startWsServerWithMessages((msg) => {
362367
if (msg.method) {
363368
methods.push(msg.method);
@@ -369,27 +374,50 @@ describe("cdp", () => {
369374
res.end("not found");
370375
return;
371376
}
372-
controller.abort(reason);
373-
res.setHeader("content-type", "application/json");
374-
res.end(
375-
JSON.stringify({
376-
webSocketDebuggerUrl: `ws://127.0.0.1:${wsPort}/devtools/browser/TEST`,
377-
}),
378-
);
377+
releaseDiscovery = () => {
378+
if (res.destroyed || res.writableEnded) {
379+
return;
380+
}
381+
res.setHeader("content-type", "application/json");
382+
res.end(
383+
JSON.stringify({
384+
webSocketDebuggerUrl: `ws://127.0.0.1:${wsPort}/devtools/browser/TEST`,
385+
}),
386+
);
387+
};
388+
markDiscoveryStarted?.();
379389
});
380390
await new Promise<void>((resolve) => {
381391
httpServer?.listen(0, "127.0.0.1", resolve);
382392
});
383393
const httpPort = (httpServer.address() as AddressInfo).port;
384394

385-
await expect(
386-
createTargetViaCdp({
387-
cdpUrl: `http://127.0.0.1:${httpPort}`,
388-
url: "https://example.com",
389-
signal: controller.signal,
395+
const pending = createTargetViaCdp({
396+
cdpUrl: `http://127.0.0.1:${httpPort}`,
397+
url: "https://example.com",
398+
signal: controller.signal,
399+
});
400+
await discoveryStarted;
401+
controller.abort(reason);
402+
403+
let cancellationDeadline: ReturnType<typeof setTimeout> | undefined;
404+
const boundedCancellation = Promise.race([
405+
pending,
406+
new Promise<never>((_resolve, reject) => {
407+
cancellationDeadline = setTimeout(
408+
() => reject(new Error("cancelled CDP discovery remained pending")),
409+
300,
410+
);
390411
}),
391-
).rejects.toBe(reason);
392-
expect(methods).toEqual([]);
412+
]);
413+
try {
414+
await expect(boundedCancellation).rejects.toBe(reason);
415+
expect(methods).toEqual([]);
416+
} finally {
417+
clearTimeout(cancellationDeadline);
418+
releaseDiscovery?.();
419+
await pending.catch(() => {});
420+
}
393421
});
394422

395423
it("reads the browser frame URL with its fragment", async () => {

extensions/browser/src/browser/cdp.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -245,7 +245,7 @@ export async function createTargetViaCdp(opts: {
245245
version = await fetchJson<{ webSocketDebuggerUrl?: string }>(
246246
appendCdpPath(discoveryUrl, "/json/version"),
247247
opts.timeouts?.httpTimeoutMs,
248-
undefined,
248+
{ signal: opts.signal },
249249
cdpControlPolicy,
250250
);
251251
} catch (err) {
Lines changed: 205 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,205 @@
1+
import { randomUUID } from "node:crypto";
2+
import fs from "node:fs/promises";
3+
import os from "node:os";
4+
import path from "node:path";
5+
import type {
6+
CodexBundleMcpThreadConfig,
7+
EmbeddedRunAttemptParamsV2 as EmbeddedRunAttemptParams,
8+
} from "openclaw/plugin-sdk/agent-harness-runtime";
9+
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
10+
import { startCodexAttemptThread } from "./attempt-startup.js";
11+
import { threadStartResult } from "./codex-app-server.test-fixtures.js";
12+
import {
13+
resolveCodexAppServerRuntimeOptions,
14+
resolveCodexComputerUseConfig,
15+
type CodexPluginConfig,
16+
} from "./config.js";
17+
import { createCodexTestHostCapabilities } from "./host-capability.test-support.js";
18+
import { defaultCodexPluginMetadataCache } from "./plugin-metadata-cache.js";
19+
import {
20+
resetCodexTestBindingStore,
21+
testCodexAppServerBindingStore,
22+
} from "./session-binding.test-helpers.js";
23+
import {
24+
clearSharedCodexAppServerClient,
25+
clearSharedCodexAppServerClientAndWait,
26+
getLeasedSharedCodexAppServerClient,
27+
} from "./shared-client.js";
28+
import { createCodexTestModel } from "./test-support.js";
29+
30+
vi.mock("./desktop-generation.js", () => ({
31+
isCodexDesktopGenerationCurrent: () => false,
32+
waitForCodexDesktopGeneration: async () => undefined,
33+
}));
34+
35+
const tempRoots = new Set<string>();
36+
37+
async function createStartupFailureFixture(
38+
mode: "transient" | "contention" | "persistent" | "unsupported",
39+
) {
40+
const root = path.join(os.tmpdir(), `openclaw-codex-startup-retry-${randomUUID()}`);
41+
tempRoots.add(root);
42+
const fixturePath = path.join(root, "startup-failure.mjs");
43+
const spawnCountPath = path.join(root, "spawn-count");
44+
const codexHome = path.join(root, "codex-home");
45+
await fs.mkdir(root, { recursive: true });
46+
await fs.writeFile(
47+
fixturePath,
48+
[
49+
'import fs from "node:fs";',
50+
'import readline from "node:readline";',
51+
"const [spawnCountPath, mode, codexHome] = process.argv.slice(2);",
52+
'const attempt = Number(fs.existsSync(spawnCountPath) ? fs.readFileSync(spawnCountPath, "utf8") : 0) + 1;',
53+
'fs.writeFileSync(spawnCountPath, String(attempt), "utf8");',
54+
"const startedAtPath = `${spawnCountPath}.started-at`;",
55+
'if (attempt === 1) fs.writeFileSync(startedAtPath, String(Date.now()), "utf8");',
56+
'const stillContended = mode === "contention" && Date.now() - Number(fs.readFileSync(startedAtPath, "utf8")) < 750;',
57+
'if (mode === "persistent" || (mode === "transient" && attempt === 1) || stillContended) {',
58+
" console.error(`Error: failed to initialize sqlite state runtime under ${codexHome}: failed to initialize state runtime at ${codexHome}`);",
59+
" process.exitCode = 1;",
60+
"} else {",
61+
" const lines = readline.createInterface({ input: process.stdin });",
62+
' lines.on("line", (line) => {',
63+
" const message = JSON.parse(line);",
64+
" if (message.id === undefined) return;",
65+
' const result = message.method === "initialize"',
66+
' ? { userAgent: `openclaw/${mode === "unsupported" ? "0.1.0" : "0.149.0"} (macOS; test)` }',
67+
` : ${JSON.stringify(threadStartResult("thread-recovered", "/repo"))};`,
68+
" process.stdout.write(`${JSON.stringify({ id: message.id, result })}\\n`);",
69+
" });",
70+
"}",
71+
].join("\n"),
72+
"utf8",
73+
);
74+
const pluginConfig = {
75+
appServer: {
76+
transport: "stdio",
77+
command: process.execPath,
78+
args: [fixturePath, spawnCountPath, mode, codexHome],
79+
requestTimeoutMs: 5_000,
80+
},
81+
} satisfies CodexPluginConfig;
82+
return { root, spawnCountPath, pluginConfig };
83+
}
84+
85+
function startFixtureAttempt(fixture: Awaited<ReturnType<typeof createStartupFailureFixture>>) {
86+
const agentDir = path.join(fixture.root, "agent");
87+
const workspaceDir = path.join(fixture.root, "workspace");
88+
const bundleMcpThreadConfig = {
89+
configPatch: undefined,
90+
diagnostics: [],
91+
evaluated: false,
92+
fingerprint: undefined,
93+
staticServerNames: [],
94+
userStaticServerNames: [],
95+
} satisfies CodexBundleMcpThreadConfig;
96+
return startCodexAttemptThread({
97+
bindingStore: testCodexAppServerBindingStore,
98+
attemptClientFactory: getLeasedSharedCodexAppServerClient,
99+
appServer: resolveCodexAppServerRuntimeOptions({ pluginConfig: fixture.pluginConfig }),
100+
pluginConfig: fixture.pluginConfig,
101+
computerUseConfig: resolveCodexComputerUseConfig({ pluginConfig: fixture.pluginConfig }),
102+
startupAuthProfileId: undefined,
103+
startupAuthBindingFingerprint: undefined,
104+
startupAuthAccountCacheKey: undefined,
105+
startupEnvApiKeyCacheKey: undefined,
106+
agentDir,
107+
config: undefined,
108+
buildAttemptParams: () =>
109+
({
110+
hostCapabilities: createCodexTestHostCapabilities(),
111+
prompt: "hello",
112+
sessionId: "session-1",
113+
sessionKey: "agent:agent-1:session-1",
114+
agentDir,
115+
sessionFile: path.join(fixture.root, "session.jsonl"),
116+
effectiveCwd: workspaceDir,
117+
workspaceDir,
118+
runId: "run-1",
119+
provider: "codex",
120+
modelId: "gpt-5.4-codex",
121+
model: createCodexTestModel("codex"),
122+
thinkLevel: "medium",
123+
disableTools: true,
124+
timeoutMs: 5_000,
125+
authStorage: {} as never,
126+
authProfileStore: { version: 1, profiles: {} },
127+
modelRegistry: {} as never,
128+
}) as EmbeddedRunAttemptParams,
129+
sessionAgentId: "agent-1",
130+
effectiveWorkspace: workspaceDir,
131+
effectiveCwd: workspaceDir,
132+
dynamicTools: [],
133+
webSearchAllowed: false,
134+
developerInstructions: undefined,
135+
finalConfigPatch: undefined,
136+
bundleMcpThreadConfig,
137+
nativeToolSurfaceEnabled: true,
138+
nativeProviderWebSearchSupport: "supported",
139+
sandboxExecServerEnabled: false,
140+
sandbox: null,
141+
contextEngineProjection: undefined,
142+
startupTimeoutMs: 10_000,
143+
signal: new AbortController().signal,
144+
onStartupTimeout: vi.fn(),
145+
spawnedBy: undefined,
146+
});
147+
}
148+
149+
describe("Codex app-server startup retry", () => {
150+
beforeEach(() => {
151+
vi.stubEnv("CODEX_API_KEY", "");
152+
vi.stubEnv("OPENAI_API_KEY", "");
153+
clearSharedCodexAppServerClient();
154+
defaultCodexPluginMetadataCache.clear();
155+
resetCodexTestBindingStore();
156+
});
157+
158+
afterEach(async () => {
159+
await clearSharedCodexAppServerClientAndWait();
160+
defaultCodexPluginMetadataCache.clear();
161+
vi.unstubAllEnvs();
162+
for (const root of tempRoots) {
163+
await fs.rm(root, { recursive: true, force: true });
164+
}
165+
tempRoots.clear();
166+
});
167+
168+
it("retries a real app-server after transient sqlite state initialization failure", async () => {
169+
const fixture = await createStartupFailureFixture("transient");
170+
const result = await startFixtureAttempt(fixture);
171+
172+
expect(result.thread.threadId).toBe("thread-recovered");
173+
expect(await fs.readFile(fixture.spawnCountPath, "utf8")).toBe("2");
174+
result.turnRoute.release();
175+
result.releaseSharedClientLease();
176+
});
177+
178+
it("waits out transient sqlite contention before retrying app-server startup", async () => {
179+
const fixture = await createStartupFailureFixture("contention");
180+
const result = await startFixtureAttempt(fixture);
181+
182+
expect(result.thread.threadId).toBe("thread-recovered");
183+
expect(await fs.readFile(fixture.spawnCountPath, "utf8")).toBe("2");
184+
result.turnRoute.release();
185+
result.releaseSharedClientLease();
186+
});
187+
188+
it("bounds retries when sqlite state initialization keeps failing", async () => {
189+
const fixture = await createStartupFailureFixture("persistent");
190+
191+
await expect(startFixtureAttempt(fixture)).rejects.toThrow(
192+
"failed to initialize sqlite state runtime",
193+
);
194+
expect(await fs.readFile(fixture.spawnCountPath, "utf8")).toBe("3");
195+
});
196+
197+
it("rejects an unsupported app-server version without retrying", async () => {
198+
const fixture = await createStartupFailureFixture("unsupported");
199+
200+
await expect(startFixtureAttempt(fixture)).rejects.toThrow(
201+
/app-server .* or newer is required/i,
202+
);
203+
expect(await fs.readFile(fixture.spawnCountPath, "utf8")).toBe("1");
204+
});
205+
});

0 commit comments

Comments
 (0)