Skip to content

refactor(server): slim the agent system prompt and move tool guidance into the tools - #2521

Merged
Dani Akash (DaniAkash) merged 4 commits into
mainfrom
refactor/slim-agent-system-prompt
Sep 2, 2026
Merged

refactor(server): slim the agent system prompt and move tool guidance into the tools#2521
Dani Akash (DaniAkash) merged 4 commits into
mainfrom
refactor/slim-agent-system-prompt

Conversation

@DaniAkash

Copy link
Copy Markdown
Contributor

Problem

apps/server/src/agent/prompt.ts was 688 lines / ~16,900 chars. It spread role, a narrated tool catalog, tool-selection tables, execution rules, per-tool error recovery, integrations, style, and repeated security reminders across many sections with heavy overlap, so the prompt was expensive and hard to reason about.

Change

Reduce the prompt to non-duplicated, cross-cutting rules and let each tool own its own guidance.

  • The regular LLM receives the system prompt plus each tool's description plus the runtime untrusted-content fence. So per-tool usage, per-tool security, and per-tool recovery belong on the tools, not the prompt.
  • Deleted from the prompt: the narrated tool catalog, the tool-selection tables, the per-tool error-recovery catalog, the final security reminder, and two dead sections (the ACP tool-namespace and soul sections were never rendered).
  • Kept in the prompt what a tool cannot own: role and mode, the trust boundary, safety, cross-tool execution workflow (multi-tab focus discipline, obstacles, retry budget), the Strata integration flow (its executable tools are remote, so their descriptions are not authored here), nudge behavior, response style, and dynamic page context.

Result: ~5,100 chars, about 70% smaller, with the same behavioral substance.

Tool changes (so the guidance still reaches the model)

  • act: prefer ref kinds over the coordinate kinds; on a failed click/fill, scroll into view and retry once; never type credentials into a page the agent navigated to itself.
  • run: note that it is for extraction and requested automation, not unsolicited page modification; and fence its output as untrusted.
  • history: fence its output as untrusted.

This also tightens security: run and history output was previously unfenced and is now wrapped in the same untrusted-content markers as read, grep, snapshot, diff, and evaluate.

Verification

Prompt tests were rewritten for the new structure and pass (34), including guards that the security, safety, integration, nudge, and mode/workspace behaviors survive and that the removed sections are gone, plus a size regression check. Browser tool tests pass (including the description-snapshot test), and typecheck and biome are clean.

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

✅ Tests passed: 1224/1227

Ran 8 of 16 suites (8 not affected by this change).

Suite Passed Failed Skipped
server-agent 193/193 0 0
server-api 195/195 0 0
server-tools 248/248 0 0
server-browser 10/10 0 0
server-integration 10/10 0 0
server-lib 142/142 0 0
server-root 38/41 0 3
agent 388/388 0 0
claw-app n/a n/a not affected
claw-onboard n/a n/a not affected
app-onboard n/a n/a not affected
build n/a n/a not affected
release n/a n/a not affected
claw-server-rust n/a n/a not affected
claw-server-rust-quality n/a n/a not affected
claw-mcp n/a n/a not affected

View workflow run

@greptile-apps

greptile-apps Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR substantially condenses the BrowserOS system prompt and relocates tool-specific guidance into browser-tool descriptions. It also adds untrusted-content wrapping to run and browser-history output, but run leaves its model-visible structured output unfenced.

  • Removes redundant prompt catalogs, selection tables, recovery sections, and unused ACP/soul sections.
  • Preserves mode, workspace, integration, navigation, safety, nudge, and response-style guidance.
  • Expands act usage and credential guidance.
  • Adds trust-boundary wrapping for run and history.

Confidence Score: 3/5

This PR should not merge until run applies the untrusted-content boundary to the actual model-visible output channel.

The built-in agent consumes run’s raw schema-backed structured result, so wrapping only its parallel text representation does not protect page-derived values, logs, or errors from being interpreted as instructions.

Files Needing Attention: packages/browseros-agent/packages/browser-mcp/src/tools/run.ts

Security Review

The run fence does not cover the structured result consumed by the built-in agent, so page-derived return values, logs, and errors can still reach the model without untrusted-content markers.

Important Files Changed

Filename Overview
packages/browseros-agent/apps/server/src/agent/prompt.ts Condenses the system prompt while retaining the principal cross-cutting mode, trust, navigation, integration, and workspace guidance.
packages/browseros-agent/apps/server/tests/agent/prompt.test.ts Reworks prompt tests around the slimmer structure, retained behavioral rules, and prompt-size target.
packages/browseros-agent/packages/browser-mcp/src/tools/act.ts Moves interaction preferences, retry guidance, and credential restrictions into the tool description.
packages/browseros-agent/packages/browser-mcp/src/tools/history.ts Fences history text as untrusted while preserving its existing structured metadata.
packages/browseros-agent/packages/browser-mcp/src/tools/history.test.ts Adds coverage that nonempty history text carries the untrusted-content marker.
packages/browseros-agent/packages/browser-mcp/src/tools/run.ts Fences only the textual representation, leaving schema-backed page-derived structured output exposed to the model without the trust marker.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  P[Page-controlled content] --> R[run value or console logs]
  R --> S[Raw structuredContent]
  R --> T[Fenced text content]
  S --> M[Built-in model]
  T -. not selected for schema-backed output .-> M
Loading
Prompt To Fix All With AI
### Issue 1
packages/browseros-agent/packages/browser-mcp/src/tools/run.ts:85-89
**Structured output bypasses trust fence**

When a requested `run` script returns or logs page-controlled instructions, only the textual representation is fenced while the schema-backed `value`, `logs`, and error fields remain raw. The built-in agent consumes that structured result, allowing hostile page content to reach the model without untrusted-content markers. **How this was verified:** The `run` output schema selects raw `structuredContent` as the model-visible result while `wrapUntrusted` is applied only to the parallel text content.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "docs: tidy prompt comments" | Re-trigger Greptile

Comment thread packages/browseros-agent/packages/browser-mcp/src/tools/run.ts
@DaniAkash
Dani Akash (DaniAkash) merged commit 192409e into main Sep 2, 2026
18 checks passed
@DaniAkash
Dani Akash (DaniAkash) deleted the refactor/slim-agent-system-prompt branch September 2, 2026 10:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant