Skip to content

Commit 77db267

Browse files
committed
feat: add tool mode selection and Codex screenshot delivery
1 parent a793e92 commit 77db267

27 files changed

Lines changed: 753 additions & 18 deletions

‎AGENTS.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -254,6 +254,8 @@ crewcoder/
254254

255255
## Current hard-won notes
256256

257+
- Tool mode defaults to `crewcoder` for new sessions; `native` uses Codex CLI or Claude SDK tools without CrewCoder built-in/extension tools. Preserve saved mode on resume and host custody precedence. Host-supplied tools, virtual filesystems and specialized agent modes force CrewCoder tools. Never apply the standalone preference around host approvals. See `crewcoder-agent/docs/TOOL_MODE.md`.
258+
257259
- ACP hosts advertise `_meta["crewcoder/questions"].version: 1` to receive `_crewcoder/request_question` with original option values, free-text/Other, secret flags, and exact provider permission details. Questions must never become automatic permission approvals. Cancel pending cards on session/provider cancellation. Explicit Never remains non-interactive; use Review when escalation should ask. See `crewcoder-agent/docs/ACP_ADAPTER.md`.
258260

259261
- Reuse provider-specific computer-use integrations where available; the universal computer-use layer is deferred. Claude's built-in CLI computer use is documented as macOS-only and interactive-only, so CrewCoder's non-interactive Agent SDK transport cannot currently reuse it. Claude Desktop Windows support does not establish SDK support; never relax strict MCP isolation as a workaround. See `crewcoder-agent/docs/CODEX_CLI_PROVIDER.md`.

‎crewcoder-agent/AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -276,7 +276,7 @@ Codex / OpenAI Responses:
276276
Claude Agent SDK:
277277
- `claude` is the local-login Agent SDK provider; `anthropic` remains the direct API-key Messages provider.
278278
- Keep `settingSources: ["project"]`, `skills: []`, and `strictMcpConfig: true`: preserve project CLAUDE.md guidance without auto-injecting global skill catalogs or undeclared MCP servers. On-demand skill files under `~/.agents/skills` and the other user catalogs remain readable; do not force Skill-tool activation.
279-
- Tool ownership is hybrid. Native Claude is limited to Read/Grep/Glob/AskUserQuestion; Bash, mutations, and specialized tools must route through the in-process CrewCoder MCP server and existing executor.
279+
- New sessions use explicit Tool mode: `crewcoder` disables native Claude coding tools and uses the CrewCoder MCP executor; `native` uses the SDK tool preset and provider permission routing. Pre-setting sessions retain the original hybrid Read/Grep/Glob/AskUserQuestion behavior. Host custody always overrides the preference. See `docs/TOOL_MODE.md`.
280280
- Disable native Read/Grep/Glob whenever a virtual `TextFileHost` is active (especially ACP SSH/SFTP); route those operations through CrewCoder MCP so the local Claude process cannot cross host boundaries.
281281
- Persist returned SDK session ids in `providerSessionIds.claude`; never substitute the CrewCoder session id or send Claude ids to another provider.
282282
- Preserve assistant/thinking deltas, native read/search tool activity, cancellation, and CrewCoder MCP tool lifecycle events.

‎crewcoder-agent/docs/CODEX_CLI_PROVIDER.md‎

Lines changed: 27 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,31 @@ See [Claude Code computer use](https://code.claude.com/docs/en/computer-use).
6060
Connecting an external automation service would be separate work; do not
6161
disable strict MCP isolation as a workaround.
6262

63+
## Image results from hosted tools
64+
65+
CrewCoder's guest/local read tool declares images on the tool result's image metadata.
66+
The Codex app-server adapter now reads those authorized local files and sends their
67+
pixels as dynamic-tool inputImage content items alongside the text description.
68+
Native view_image remains disabled in hosted-tools-only sessions. ACP client text
69+
I/O and hosted-text-only tools do not gain a binary filesystem fallback.
70+
71+
The adapter rechecks workspace and session external-directory scope, sniffs PNG,
72+
JPEG, GIF or WebP bytes, and limits a response to four images, 8 MiB per image and
73+
16 MiB total. Missing, changed, unsupported or oversized images produce a failed
74+
tool response with a readable delivery error, without partial image content.
75+
Image bytes are encoded only on the app-server wire; durable tool records retain
76+
paths and metadata.
77+
78+
Regression command from the repository root:
79+
80+
```sh
81+
node node_modules/vitest/vitest.mjs run --config test/codex-tool-images.config.ts
82+
```
83+
84+
A guest with an older CrewCoder bundle still returns only the binary description.
85+
Build and install the updated bundle in that guest before repeating VM GUI acceptance;
86+
restarting the desktop application alone does not update an existing guest disk.
87+
6388
## No fallback
6489

6590
If the app-server cannot start, the turn fails with the cause. It never falls back to the direct Responses transport, because that would need CrewCoder-owned credentials.
@@ -69,7 +94,7 @@ If the app-server cannot start, the turn fails with the cause. It never falls ba
6994

7095
## Hosted-tools-only lockdown
7196

72-
When CrewCoder routes the filesystem itself (`useProviderNativeFileTools: false`, which ACP virtual-filesystem sessions and Supervisor hosted sessions set), only CrewCoder's dynamic tools may act. The launch adds:
97+
When `toolMode: crewcoder` is selected, or `useProviderNativeFileTools: false` enforces ACP virtual-filesystem or Supervisor custody, only CrewCoder's dynamic tools may act. The launch adds:
7398

7499
| Control | Why |
75100
| --- | --- |
@@ -97,4 +122,4 @@ Fail-closed rules:
97122
## Limits
98123

99124
- Feature names are tied to the Codex versions above. A Codex update that adds a new native tool under a new feature name is not removed until it is added here. The read-only sandbox still blocks native writes and command execution.
100-
- Without lockdown (plain CrewCoder use), Codex keeps its native tools and the user's full Codex config, as when running `codex` directly.
125+
- With native Tool mode, Codex keeps its native tools and the user's full Codex config, as when running `codex` directly. New standalone sessions default to CrewCoder tools. See [Tool mode](TOOL_MODE.md).

‎crewcoder-agent/docs/TOOL_MODE.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# Tool mode
2+
3+
Tool mode selects who supplies coding tools for a new standalone CrewCoder session.
4+
The default is `crewcoder`.
5+
6+
| Mode | Tool ownership |
7+
| --- | --- |
8+
| `crewcoder` | CrewCoder built-in tools and enabled trusted extension tools, using CrewCoder execution and approvals. |
9+
| `native` | Codex CLI or Claude Agent SDK native tools. CrewCoder built-in and extension tools are omitted. |
10+
11+
Choose **Settings > /tool-mode** in the command palette, or use:
12+
13+
```text
14+
/tool-mode crewcoder
15+
/tool-mode native
16+
/tool-mode status
17+
```
18+
19+
The equivalent backend setting is:
20+
21+
```sh
22+
crewcoder config set toolMode crewcoder
23+
crewcoder config set toolMode native
24+
```
25+
26+
The setting applies to new sessions. Use `/new` after changing it. A saved session
27+
retains its tool mode on resume even when the global default changes. Branches
28+
inherit their source session's mode. Sessions created before this setting retain
29+
their previous tool behavior. Changing provider does not change the saved mode.
30+
31+
## Provider support
32+
33+
Codex CLI supports both modes. CrewCoder mode uses the same native feature and
34+
MCP lockdown as hosted sessions, including a read-only native sandbox as a
35+
backstop for native writes. CrewCoder tools still read, execute and write through
36+
their normal executor and CrewCoder approval policy.
37+
38+
Claude supports both modes. CrewCoder mode disables native Read/Grep/Glob and
39+
routes coding actions through CrewCoder's MCP tools. Native mode uses the SDK's
40+
`claude_code` tool preset and forwards SDK permission requests to the interactive
41+
host. Missing, cancelled or unrecognized permission answers decline the request.
42+
Never mode declines requests requiring interactive approval; Full access permits
43+
them. Native mode does not remove strict MCP isolation or enable unsupported
44+
Claude computer use. Claude native mode rejects CrewCoder's OS Sandboxed approval
45+
mode because the adapter does not implement that sandbox.
46+
47+
API-only providers, including OpenCode's Messages transport, cannot supply a
48+
native tool runtime. Selecting native mode with these providers reports an error
49+
before inference. Choose CrewCoder tools or a supported native provider.
50+
51+
Generic process, model-command and nested ACP providers cannot enforce CrewCoder
52+
tool ownership, so new CrewCoder-tool sessions reject these transports before
53+
spawning. Pre-setting sessions retain their prior transport behavior.
54+
55+
## Host policy and specialized modes
56+
57+
Host policy takes precedence over this preference. Hosted-only sessions, virtual
58+
filesystems, supplied tool registries, and host-added tools require CrewCoder
59+
tools. A standalone preference cannot enable a second native filesystem or GUI
60+
path around CrewMate's grants, approvals, VM routing or takeover controls.
61+
62+
The `crewcoder` approved-plan workflow and the `plugin` and `extension` agent
63+
modes also require CrewCoder tools. Native tools cannot enforce the plan's
64+
mutation gate, and the architect modes depend on CrewCoder's specialized tools.
65+
66+
Tool mode does not change logins, provider/model selection, conversation storage,
67+
or TUI commands such as sessions, settings, branching and compaction. Enabled
68+
CrewCoder context hooks and prompt contributions retain their existing behavior.
69+
Automatic CrewCoder mutation checkpoints, extension tool interception, and
70+
CrewCoder file-change recording apply to CrewCoder tools. Native actions follow
71+
their provider's execution and reporting behavior.
72+
73+
## Verification
74+
75+
From the repository root:
76+
77+
```sh
78+
node node_modules/vitest/vitest.mjs run --config test/tool-mode.config.ts
79+
npm run typecheck -w @onpoint-dev-tools/crewcoder-agent
80+
npm run typecheck -w @onpoint-dev-tools/crewcoder-tui
81+
```
82+
83+
The focused suite covers config validation, session pinning, pre-setting sessions,
84+
host precedence, hosted writes, unsupported providers, Codex process flags and
85+
permission responses, Claude SDK tool selection and permissions, and the TUI
86+
picker and persistence failures. These are automated adapter and rendering checks;
87+
live provider and terminal acceptance is separate.

‎crewcoder-agent/src/cli.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1641,7 +1641,7 @@ program.command("config").argument("<action>", "show or set").argument("[key]").
16411641
}
16421642
if (action !== "set") throw new Error("Supported actions: show, set.");
16431643
if (!key || value === undefined) throw new Error("Usage: config set <key> <value>");
1644-
const allowed = ["defaultMode", "defaultProvider", "defaultModel", "workerIdentityEnabled", "maxIterations", "stallDetection", "stallRepeatThreshold", "stallErrorThreshold", "allowExtensionTools", "allowExtensionHooks", "allowExtensionModules", "allowExtensionLiveUi", "disabledExtensions", "trustedExtensions", "sandboxedExtensions", "sandboxAllowedHosts", "sandboxNetworkIsolation", "checkpointsEnabled", "autoCompact", "autoCompactThresholdTokens", "compactionPreview", "goals.maxTurns", "goals.checkModel", "goals.timeoutMinutes"] as const;
1644+
const allowed = ["toolMode", "defaultMode", "defaultProvider", "defaultModel", "workerIdentityEnabled", "maxIterations", "stallDetection", "stallRepeatThreshold", "stallErrorThreshold", "allowExtensionTools", "allowExtensionHooks", "allowExtensionModules", "allowExtensionLiveUi", "disabledExtensions", "trustedExtensions", "sandboxedExtensions", "sandboxAllowedHosts", "sandboxNetworkIsolation", "checkpointsEnabled", "autoCompact", "autoCompactThresholdTokens", "compactionPreview", "goals.maxTurns", "goals.checkModel", "goals.timeoutMinutes"] as const;
16451645
if (!allowed.includes(key as typeof allowed[number])) throw new Error(`Supported keys: ${allowed.join(", ")}`);
16461646
const config = setConfigValue(key as typeof allowed[number], value);
16471647
console.log(JSON.stringify(config, null, 2));

‎crewcoder-agent/src/core/agent-loop-continue.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,8 @@ export async function runAgentLoopContinue(input: {
6060
images: input.images
6161
}, {
6262
...options,
63+
// A changed global default must not change the tool authority of a resumed session.
64+
toolMode: session.toolMode ?? "legacy",
6365
sessionId: input.sessionId,
6466
resumeFromSessionId: input.sessionId,
6567
resumeContext: session.pendingResumeContext,

‎crewcoder-agent/src/core/agent-loop.ts‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ import { inspectProject, formatProjectInspection, type ProjectInspection } from
3434
import { prepareLiveCompaction, applyCompactionProposal, summarizeMessagesForHandoff, type SessionCompaction } from "./session-compaction.js";
3535
import { createSessionCheckpoint, MAX_SESSION_CHECKPOINTS, type SessionCheckpoint } from "./session-checkpoints.js";
3636
import { readConfig } from "./config.js";
37+
import { resolveToolMode, type ToolMode } from "./tool-mode.js";
3738
import { resolveActiveWorker, buildIdentityPrompt } from "./identity.js";
3839
import { dumpModelInput } from "./model-input-dump.js";
3940
import { appendAuditLog } from "./audit-log.js";
@@ -55,6 +56,8 @@ import {
5556
} from "../modes/crewcoder-workflow.js";
5657

5758
export type AgentLoopOptions = {
59+
/** Session tool selection; host custody takes precedence. */
60+
toolMode?: ToolMode | "legacy";
5861
/** Host policy: only supplied text tools, without extension hooks or local project discovery. */
5962
hostedToolsOnly?: boolean;
6063
modelClient?: ModelClient;
@@ -200,6 +203,7 @@ export async function runAgentLoop(request: AgentRequest, options: AgentLoopOpti
200203
if (options.hostedToolsOnly && (!options.tools || !options.textFiles?.readTextFile || !options.textFiles.writeTextFile)) throw new Error("Hosted tool policy requires supplied tools and both host text methods");
201204
const sessionId = options.sessionId ?? createSessionId();
202205
const runtimeConfig = readConfig();
206+
const virtualFilesystem = options.hostedToolsOnly || (options.virtualFilesystem ?? options.textFiles !== undefined);
203207
const integrationProfile = options.integrationProfile ?? resolveIntegrationProfile(request.cwd, runtimeConfig);
204208
const mode = options.hostedToolsOnly ? "general" : resolveMode(request.requestedMode);
205209
if (mode === "plugin" && integrationProfile !== "crewcode") {
@@ -244,17 +248,17 @@ export async function runAgentLoop(request: AgentRequest, options: AgentLoopOpti
244248
let budgetWarningEmitted = typeof tokenBudget === "number" && tokenBudgetStatus(usageSummary, tokenBudget).warningReached;
245249
let budgetDownshiftRequested = false;
246250
let budgetExceeded = typeof tokenBudget === "number" && tokenBudgetStatus(usageSummary, tokenBudget).exceeded;
247-
const builtInTools = withCrewcoderWorkflowTools(options.tools ?? createToolRegistry(integrationProfile, mode), mode);
251+
const toolMode = resolveToolMode(options.toolMode ?? runtimeConfig.toolMode, { ...options, virtualFilesystem, suppliedTools: options.tools, mode });
252+
const builtInTools = withCrewcoderWorkflowTools(options.tools ?? (toolMode === "native" ? [] : createToolRegistry(integrationProfile, mode)), mode);
248253
// Host-executed tools stay available under the hosted policy: the host, not CrewCoder, authorizes them.
249254
const tools = options.hostedToolsOnly ? [...builtInTools.filter(tool => tool.name === "read" || tool.name === "write"), ...(options.additionalTools ?? [])] : options.tools
250255
? [...builtInTools, ...(options.additionalTools ?? [])]
251-
: [...builtInTools, ...(await loadTrustedExtensionTools()), ...(options.additionalTools ?? [])];
256+
: [...builtInTools, ...(toolMode === "native" ? [] : await loadTrustedExtensionTools()), ...(options.additionalTools ?? [])];
252257
const crewcoderWorkflow = mode === "crewcoder"
253258
? applyIncomingUserMessage(options.initialCrewcoderWorkflow ?? reconstructCrewcoderWorkflow(initialMessages), request.prompt)
254259
: undefined;
255260
const modelClient = options.modelClient ?? createModelClientFromEnv();
256261
const approvalMode = options.approvalMode ?? "never";
257-
const virtualFilesystem = options.hostedToolsOnly || (options.virtualFilesystem ?? options.textFiles !== undefined);
258262
// 0/undefined means unlimited. A working agent is bounded by the task, by an
259263
// opt-in token budget, or by stall detection — never by a turn counter.
260264
const requestedIterations = options.maxIterations ?? runtimeConfig.maxIterations;
@@ -332,7 +336,7 @@ export async function runAgentLoop(request: AgentRequest, options: AgentLoopOpti
332336
const skills = selectSkills(mode, request.prompt);
333337
const docs = selectDocs(mode);
334338
const crewTasksConfig = readCrewTasksConfig();
335-
const crewTasksPrompt = crewTasksConfig.enabled && !options.hostedToolsOnly
339+
const crewTasksPrompt = crewTasksConfig.enabled && !options.hostedToolsOnly && toolMode !== "native"
336340
? [
337341
"crew-tasks is enabled for this project/session.",
338342
"For complex multi-step work, use TaskCreate/TaskList/TaskGet/TaskUpdate to maintain persistent project tasks in .crewcoder/tasks.",
@@ -348,6 +352,7 @@ export async function runAgentLoop(request: AgentRequest, options: AgentLoopOpti
348352
const selectedSystemPromptName = options.systemPromptName;
349353
const selectedSystemPrompt = selectedSystemPromptName ? getSystemPrompt(selectedSystemPromptName) : null;
350354
const defaultSystemPrompt = [
355+
...(toolMode ? [`Tool mode: ${toolMode}. ${toolMode === "native" ? "Use the provider's native tools. CrewCoder built-in and extension tools are unavailable." : "Use the tools supplied by CrewCoder for actions."}`] : []),
351356
options.hostedToolsOnly ? "You operate through host-authorized virtual text records only. Delegate execution through the host management bridge. Local project files, shell commands, native provider tools, and extension execution are unavailable." : buildSystemPrompt({ mode, skills, docs, identityPrompt: workerIdentityEnabled && activeWorker ? buildIdentityPrompt(activeWorker) : null, crewTasksPrompt, extensionContext }),
352357
formatExternalDirectories(externalDirectories)
353358
].filter(Boolean).join("\n\n");
@@ -382,6 +387,7 @@ export async function runAgentLoop(request: AgentRequest, options: AgentLoopOpti
382387
contextWindow: usageSummary.contextWindow,
383388
approvalMode,
384389
modelClient,
390+
toolMode: toolMode ?? "legacy",
385391
// Unlimited unless the parent explicitly caps it. Child workers are
386392
// bounded by stall detection and maxChildWorkerDepth, not a turn count.
387393
maxIterations: delegation.maxIterations,
@@ -437,6 +443,7 @@ export async function runAgentLoop(request: AgentRequest, options: AgentLoopOpti
437443
provider: options.providerId,
438444
model: options.model,
439445
effort: options.effort,
446+
toolMode,
440447
events,
441448
messages,
442449
modelTurns,
@@ -624,6 +631,7 @@ export async function runAgentLoop(request: AgentRequest, options: AgentLoopOpti
624631
// A file host may also represent local unsaved buffers, so capability
625632
// presence alone is not enough to classify the workspace as virtual.
626633
useProviderNativeFileTools: !virtualFilesystem,
634+
toolMode,
627635
approvalMode,
628636
// Native-session providers must use the same context policy as the
629637
// outer CrewCoder loop or they can compact first at a smaller default.

0 commit comments

Comments
 (0)