Skip to content

Commit 7bb150f

Browse files
committed
Merge upstream main into dustywusty/local-eval-on-upstream
Takes main's engine choice (#831): Docker Desktop is opened only when it has run before, and each installation stays on the engine it was set up on. The old copy of #831 on this branch is dropped, including after_engine_install, which duplicated main's answering_after_install and ignored the saved engine. Kept from this branch: path_with_docker, downloads chosen by the machine's architecture on Windows on ARM, and the per-user Docker Desktop location on Windows. Takes main's #823 and #821 (StartFresh replaces DatabaseReset; the local Intelligence screen offers it too), #860, and #858's Recorder step label. The web app keeps main's theme and skin picker (#856); this branch's CopilotKit reskin of the app and its two fonts are dropped. Its desktop CSS (room for the macOS window buttons, sidebar resize) stays.
2 parents b6253bc + 4766e42 commit 7bb150f

833 files changed

Lines changed: 248810 additions & 8949 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.claude/skills/openbot-copy/SKILL.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,8 @@ because the admin is the one who sets them. A page anybody can open may not.
4040

4141
1. **Say what happened, then what to do.** "Gmail isn't connected. Connect it to let Bots read your
4242
mail." Not why the screen is built the way it is.
43-
2. **One idea per sentence. Short sentences.** A description is one or two sentences, about 20 words.
43+
2. **Use complete sentences for explanations.** Page descriptions, helper text, notices, and empty-state bodies need a subject and a verb, or a direct instruction ("Connect your accounts"). Avoid fragments such as "The apps your Bots use" or "What this Bot can do." Headings, labels, buttons, and short status labels may be fragments.
44+
**Keep only useful explanation.** State the action, outcome, or constraint the person needs. Remove copy that merely repeats the heading. Use natural sentence lengths; do not chop prose into fragments to meet a word count.
4445
3. **Plain words.** "use", not "hold a grant on". "can't", not "is unable to". Contractions are fine.
4546
4. **No "rather than", no em dashes, no "which is why", no "so that nothing … that may be wrong".**
4647
These are the marks of text that is explaining itself. Cut the clause or make it its own sentence.
@@ -59,6 +60,7 @@ because the admin is the one who sets them. A page anybody can open may not.
5960

6061
| Before | After |
6162
|---|---|
63+
| The apps and accounts your Bots use as you. Open one to connect it or change it. | Connect your accounts so Bots can use apps on your behalf. |
6264
| Your connected accounts could not be loaded, so nothing is listed here rather than a list that may be wrong. Reload the page, and tell an administrator if it persists. | Couldn't load your connections. Reload the page, or ask an admin if it keeps happening. |
6365
| Sent to the coworker it is for. Type @ to choose one yourself. | OpenBot picks the right Bot. Type @ to choose one. |
6466
| Gmail is a Composio app, and this deployment has no Composio key. (to a non-admin) | Gmail isn't set up yet. An admin has to turn it on. |
@@ -73,5 +75,6 @@ because the admin is the one who sets them. A page anybody can open may not.
7375
| "this deployment" on a page anybody can open | OpenBot, or say what is missing |
7476
| A sentence over ~25 words | Split it, or cut the clause that justifies the screen |
7577
| "rather than", an em dash, "which is" in UI text | Rewrite as two short sentences |
78+
| A description beginning "The apps…", "What…", "Who…", or "Where…" without a main verb | State the action or behavior in a complete sentence; remove redundant explanation |
7679
| An env var, OAuth, Composio or "grant" outside Admin | Say what the person sees |
7780
| A test asserting long explanatory copy | Assert the short sentence, or the role and the key words |

‎.env.example‎

Lines changed: 30 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -129,12 +129,13 @@ INTELLIGENCE_GATEWAY_WS_URL=wss://realtime.intelligence.copilotkit.ai
129129
#
130130
# npx --yes copilotkit@latest login # browser sign-in
131131
# npx --yes copilotkit@latest project select # writes the cpk-... key to .env as CPK_INTELLIGENCE_API_KEY
132-
# bun scripts/setup-learning.ts # copies it here, as INTELLIGENCE_API_KEY
132+
# bun scripts/setup-learning.ts # ensures the Learning container exists
133133
#
134-
# The server reads only INTELLIGENCE_API_KEY, so a key left as CPK_INTELLIGENCE_API_KEY is not seen.
134+
# OpenBot reads CPK_INTELLIGENCE_API_KEY directly. INTELLIGENCE_API_KEY is a legacy fallback.
135+
# When both are set, the non-empty CPK_INTELLIGENCE_API_KEY takes precedence.
135136
#
136137
# The runtime key is also under "API Keys" in your project at https://intelligence.copilotkit.ai
137-
INTELLIGENCE_API_KEY=
138+
CPK_INTELLIGENCE_API_KEY=
138139

139140
# Optional, and blank on purpose. Managed Intelligence derives entitlement from the project key
140141
# above and issues no licence token. A self-hosted Intelligence with its own licence sets this and
@@ -166,6 +167,11 @@ COPILOTKIT_LICENSE_TOKEN=
166167
# CPK_INTELLIGENCE_SKILLS_REVISION=exact-revision-id
167168
# See docs/automatic-learning.md for managed/self-hosted setup and permissions.
168169

170+
# OpenBot accepts the OpenBot Recorder extension only from its own extension id. A fork, or a build
171+
# signed with another key, has another id: list it here, comma separated, each 32 letters from a to
172+
# p. Any other value stops the server at boot.
173+
# OPENBOT_RECORDER_EXTRA_EXTENSION_IDS=
174+
169175
# How long a Bot's stream may say nothing before this deployment gives up on the turn, in
170176
# milliseconds. A Bot is any AG-UI endpoint, which means it will be redeployed mid-answer, its own
171177
# upstream will time out, and it will sometimes accept a connection and then write nothing at all.
@@ -207,6 +213,14 @@ OPENAI_API_KEY=
207213
# BOT_MODEL=openai/gpt-5.5
208214
#
209215
# OPENAI_BASE_URL=
216+
# Behind an OPENAI_BASE_URL that is not OpenAI or Azure, built-in Bots call the endpoint's Chat
217+
# Completions API, which is what most compatible endpoints serve, unless their model takes tools only
218+
# through the Responses API (gpt-5.6 and later, the -pro models, gpt-5.3-codex, under a gateway's
219+
# `openai/` name too): those call `/responses`, which a gateway or proxy in front of OpenAI serves.
220+
# A gateway that serves the Responses API keeps Responses-only features, such as reasoning summaries,
221+
# for every model with `responses`. `chat` forces Chat Completions for every model; any other value is
222+
# an error when a built-in Bot runs.
223+
# COPILOTKIT_OPENAI_API=responses
210224

211225
# Optional dictation, independent of every agent/chat provider. Set provider, base URL,
212226
# and model together. The endpoint must implement POST /audio/transcriptions under this
@@ -367,6 +381,16 @@ COMPUTER_TOKEN=
367381
# Optional absolute app-owned data root; defaults to the platform's OpenBot user-data directory.
368382
# OPENBOT_LOCAL_COMPUTER_DIR=
369383

384+
# The OpenBot browser: each person's own browser, opened from OpenBot with the Recorder already in it
385+
# and connected. Unset, the supervisor below gives it a computer of its own (on a computer in the cloud
386+
# it reaches the open web, this OpenBot and the Intelligence gateway, nothing inside the deployment);
387+
# the one computer every Bot shares never runs it. Set, it runs on this
388+
# computer instead, as a window on the person's own desktop, while Bots' computers stay where they are:
389+
# OpenBot Desktop points it at its host-side local Chrome computer, called with COMPUTER_TOKEN. The
390+
# Recorder in it reaches OpenBot at OPENBOT_PUBLIC_URL (or BETTER_AUTH_URL), which must be https, or
391+
# http on 127.0.0.1 or [::1] with a port.
392+
# OPENBOT_BROWSER_URL=http://127.0.0.1:4101
393+
370394
# Which Bot this computer belongs to, and therefore which profile directory it uses.
371395
# COMPUTER_BOT_ID=shared
372396

@@ -426,7 +450,9 @@ MANAGED_AGENT_TOKEN=
426450

427451
# One computer per Bot. Unset, every Bot shares the computer at AGENT_COMPUTER_URL, suitable on a
428452
# laptop and explicit about being one shared machine. Set, the supervisor gives each Bot a container
429-
# of its own, with its own /workspace and its own browser profile.
453+
# of its own, with its own /workspace and its own browser profile. A person's own OpenBot browser
454+
# runs only on a computer of its own, so it needs this (or OPENBOT_BROWSER_URL above): on the shared
455+
# computer a Bot's shell could read the person's logins off the disk.
430456
COMPUTER_SUPERVISOR_URL=
431457
# Shared with the supervisor. Not the security boundary, its four verbs are, but it keeps anything
432458
# else that can reach the port from cycling a Bot's computer.

‎.github/CODEOWNERS‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,4 +13,4 @@
1313
# Adding a name here grants both powers. Add people who know this codebase well
1414
# enough to say no.
1515

16-
* @davidmckayv @guidovizoso @tylerslaton @MikeRyanDev @mxmzb
16+
* @davidmckayv @guidovizoso @tylerslaton @MikeRyanDev @mxmzb @dustywusty @samjulien

‎.github/workflows/ci.yml‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -132,14 +132,16 @@ jobs:
132132
- run: bun install --frozen-lockfile
133133
- run: bun install --frozen-lockfile
134134
working-directory: agent-computer
135-
- run: bunx playwright install --with-deps chromium
135+
# `--no-shell` as the image does, so the headless run below proves nothing needs the shell.
136+
- run: bunx playwright install --with-deps --no-shell chromium
136137
working-directory: agent-computer
137138
# Headless here: these launch through the profile path without the computer's own X server.
138139
# The headless run of the Web Bot Auth test is also the one that checks the user agent fix.
139140
- run: >-
140141
bun test tests/sign-in.test.ts tests/session-cookies.test.ts tests/follows-popup.test.ts
141142
tests/demonstration-live.test.ts tests/webrtc-proxy.test.ts tests/web-bot-auth-browser.test.ts
142-
tests/handoff-page-restore.test.ts tests/enter-text.test.ts
143+
tests/handoff-page-restore.test.ts tests/enter-text.test.ts tests/person-browser.test.ts tests/person-browser-computer.test.ts
144+
tests/screencast-keys-browser.test.ts
143145
working-directory: agent-computer
144146
env:
145147
OPENBOT_COMPUTER_BROWSER: '1'
@@ -394,7 +396,7 @@ jobs:
394396
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
395397
with:
396398
persist-credentials: false
397-
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
399+
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
398400
with:
399401
python-version: "3.12"
400402
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0

‎.github/workflows/desktop-signing.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,7 @@ jobs:
9393
- uses: dtolnay/rust-toolchain@683b588c487a69d01f6c6ddf40f036ff682cc2c8 # master
9494
with:
9595
toolchain: stable
96-
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
96+
- uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3
9797
with:
9898
workspaces: desktop/src-tauri
9999
key: windows-signing

‎.github/workflows/desktop.yml‎

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,10 @@ name: Desktop
99
# it at runtime with an `expect`, so a bad row panics the window rather than failing a type check.
1010
on:
1111
pull_request:
12-
paths: ["desktop/**", "shared/harnesses.json", "shared/marks/**", "package.json", ".github/workflows/desktop.yml"]
12+
paths: ["desktop/**", "shared/harnesses.json", "shared/marks/**", "shared/recorder-extension.json", "package.json", ".github/workflows/desktop.yml"]
1313
push:
1414
branches: [main]
15-
paths: ["desktop/**", "shared/harnesses.json", "shared/marks/**", "package.json", ".github/workflows/desktop.yml"]
15+
paths: ["desktop/**", "shared/harnesses.json", "shared/marks/**", "shared/recorder-extension.json", "package.json", ".github/workflows/desktop.yml"]
1616
workflow_call:
1717
secrets:
1818
OPENBOT_GOOGLE_MODEL_OAUTH_CLIENT_SECRET:
@@ -39,7 +39,7 @@ jobs:
3939
with:
4040
toolchain: stable
4141
components: rustfmt, clippy
42-
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
42+
- uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3
4343
with:
4444
workspaces: desktop/src-tauri
4545
# Linux is the only target needing system libraries, and it needs them even to check.
@@ -78,7 +78,7 @@ jobs:
7878
- uses: dtolnay/rust-toolchain@683b588c487a69d01f6c6ddf40f036ff682cc2c8 # master
7979
with:
8080
toolchain: stable
81-
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
81+
- uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3
8282
with:
8383
workspaces: desktop/src-tauri
8484
key: ${{ matrix.platform.name }}
@@ -143,6 +143,18 @@ jobs:
143143
- name: Rust regression tests
144144
run: cargo test --locked --lib --bins
145145
working-directory: desktop/src-tauri
146+
# The binary run as the Recorder's browsers run it: the native messaging host Chrome starts, and
147+
# the install that writes its manifests. The host name and extension id come from
148+
# shared/recorder-extension.json, which the Rust reads at compile time.
149+
- name: Recorder native host tests
150+
run: cargo test --locked --test recorder_host
151+
working-directory: desktop/src-tauri
152+
# Only a release build is a GUI-subsystem executable on Windows, and only a release build proves
153+
# the debug-only test seam is absent.
154+
- name: Recorder native host tests (release)
155+
if: matrix.platform.name == 'windows'
156+
run: cargo test --locked --release --test recorder_host
157+
working-directory: desktop/src-tauri
146158
# Hosted Windows runners are administrators. Exercise setup again with an actual Users-only
147159
# account: DISM's feature reads worked as CI's administrator but refused ordinary app users.
148160
- name: Windows standard-user setup regression

‎.github/workflows/publish-release.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -133,7 +133,7 @@ jobs:
133133
ref: ${{ github.sha }}
134134
persist-credentials: false
135135
- uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
136-
- uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
136+
- uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
137137
with:
138138
registry: ghcr.io
139139
username: ${{ github.actor }}
@@ -198,7 +198,7 @@ jobs:
198198
ref: ${{ github.sha }}
199199
persist-credentials: false
200200
- uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
201-
- uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
201+
- uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
202202
with:
203203
registry: ghcr.io
204204
username: ${{ github.actor }}
@@ -271,7 +271,7 @@ jobs:
271271
image: ${{ fromJSON(needs.verify.outputs.components) }}
272272
steps:
273273
- uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
274-
- uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
274+
- uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
275275
with:
276276
registry: ghcr.io
277277
username: ${{ github.actor }}

‎.github/workflows/security_zizmor.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ jobs:
4949
- name: Run zizmor
5050
# `min-severity: low` blocks on the broadest set of findings without flagging
5151
# hypothetical-only informational notes.
52-
uses: zizmorcore/zizmor-action@85b2b9fb8e3325e7a23d7d0a9d7ba5ae0f999e3e # v0.7.1
52+
uses: zizmorcore/zizmor-action@53290badfa369cb0bf7ad16342cb62c2fa47ac7b # v0.8.0
5353
with:
5454
min-severity: low
5555
advanced-security: false

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,3 +53,6 @@ venv/
5353
.pytest_cache/
5454
.ruff_cache/
5555
.mypy_cache/
56+
57+
# The Recorder extension's signing key never enters the repository (Decision 9).
58+
extension/**/*.pem

0 commit comments

Comments
 (0)