refactor(server): slim the agent system prompt and move tool guidance into the tools - #2521
Conversation
✅ Tests passed: 1224/1227Ran 8 of 16 suites (8 not affected by this change).
|
Greptile SummaryThe PR substantially condenses the BrowserOS system prompt and relocates tool-specific guidance into browser-tool descriptions. It also adds untrusted-content wrapping to
Confidence Score: 3/5This PR should not merge until The built-in agent consumes Files Needing Attention: packages/browseros-agent/packages/browser-mcp/src/tools/run.ts
|
| Filename | Overview |
|---|---|
| packages/browseros-agent/apps/server/src/agent/prompt.ts | Condenses the system prompt while retaining the principal cross-cutting mode, trust, navigation, integration, and workspace guidance. |
| packages/browseros-agent/apps/server/tests/agent/prompt.test.ts | Reworks prompt tests around the slimmer structure, retained behavioral rules, and prompt-size target. |
| packages/browseros-agent/packages/browser-mcp/src/tools/act.ts | Moves interaction preferences, retry guidance, and credential restrictions into the tool description. |
| packages/browseros-agent/packages/browser-mcp/src/tools/history.ts | Fences history text as untrusted while preserving its existing structured metadata. |
| packages/browseros-agent/packages/browser-mcp/src/tools/history.test.ts | Adds coverage that nonempty history text carries the untrusted-content marker. |
| packages/browseros-agent/packages/browser-mcp/src/tools/run.ts | Fences only the textual representation, leaving schema-backed page-derived structured output exposed to the model without the trust marker. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart LR
P[Page-controlled content] --> R[run value or console logs]
R --> S[Raw structuredContent]
R --> T[Fenced text content]
S --> M[Built-in model]
T -. not selected for schema-backed output .-> M
Prompt To Fix All With AI
### Issue 1
packages/browseros-agent/packages/browser-mcp/src/tools/run.ts:85-89
**Structured output bypasses trust fence**
When a requested `run` script returns or logs page-controlled instructions, only the textual representation is fenced while the schema-backed `value`, `logs`, and error fields remain raw. The built-in agent consumes that structured result, allowing hostile page content to reach the model without untrusted-content markers. **How this was verified:** The `run` output schema selects raw `structuredContent` as the model-visible result while `wrapUntrusted` is applied only to the parallel text content.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Reviews (1): Last reviewed commit: "docs: tidy prompt comments" | Re-trigger Greptile
Problem
apps/server/src/agent/prompt.tswas 688 lines / ~16,900 chars. It spread role, a narrated tool catalog, tool-selection tables, execution rules, per-tool error recovery, integrations, style, and repeated security reminders across many sections with heavy overlap, so the prompt was expensive and hard to reason about.Change
Reduce the prompt to non-duplicated, cross-cutting rules and let each tool own its own guidance.
descriptionplus the runtime untrusted-content fence. So per-tool usage, per-tool security, and per-tool recovery belong on the tools, not the prompt.Result: ~5,100 chars, about 70% smaller, with the same behavioral substance.
Tool changes (so the guidance still reaches the model)
act: prefer ref kinds over the coordinate kinds; on a failed click/fill, scroll into view and retry once; never type credentials into a page the agent navigated to itself.run: note that it is for extraction and requested automation, not unsolicited page modification; and fence its output as untrusted.history: fence its output as untrusted.This also tightens security:
runandhistoryoutput was previously unfenced and is now wrapped in the same untrusted-content markers asread,grep,snapshot,diff, andevaluate.Verification
Prompt tests were rewritten for the new structure and pass (34), including guards that the security, safety, integration, nudge, and mode/workspace behaviors survive and that the removed sections are gone, plus a size regression check. Browser tool tests pass (including the description-snapshot test), and typecheck and biome are clean.