Skip to content

Commit 131c0bd

Browse files
YanxRealYanxReal
committed
feat: Vision/UI — 4 new tools (47 total) + docs 47
- simctl_get_screen_analysis: xcrun simctl io screenshot + sips -g pixelWidth/Height → {imagePath,resolution,scale} for Vision LLM - simctl_inspect_ui_tree: AppleScript temp file via System Events entire contents window 1 → JSON [{role,name,title,value,center}] - simctl_tap_by_text: case-insensitive partial/exact search via escaped AppleScript, click center, returns CLICKED:x,y:role - simctl_fill_field: locate text field by label/placeholder, clearFirst (cmd+a+del), keystroke type, role check - index.js 43→47, 1.2.0→1.3.0, README/docs 47, SECURITY validated, sips+osascript temp file sanitization
1 parent 72829db commit 131c0bd

8 files changed

Lines changed: 601 additions & 31 deletions

File tree

‎README.es.md‎

Lines changed: 16 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,14 @@
44

55
**Servidor Model Context Protocol para el ecosistema Apple**
66

7-
*Conecta OpenCode, Codex y Claude Code con Xcode — 43 herramientas profesionales en un solo `index.js`*
7+
*Conecta OpenCode, Codex y Claude Code con Xcode — 47 herramientas profesionales en un solo `index.js`*
88

99
[![CI](https://github.com/YanxReal/Xcode-MPC/actions/workflows/ci.yml/badge.svg)](https://github.com/YanxReal/Xcode-MPC/actions)
1010
[![Node >=18](https://img.shields.io/badge/node-%3E%3D18-339933?logo=node.js&logoColor=white)](https://nodejs.org)
1111
[![Yarn 4](https://img.shields.io/badge/yarn-4.18-2C8EBB?logo=yarn&logoColor=white)](https://yarnpkg.com)
1212
[![MCP](https://img.shields.io/badge/MCP-Stdio_Transport-7B68EE)](https://modelcontextprotocol.io)
1313
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
14-
[![Version](https://img.shields.io/badge/version-1.2.0-blue.svg)](package.json)
14+
[![Version](https://img.shields.io/badge/version-1.3.0-blue.svg)](package.json)
1515

1616
> 🌐 **Idioma:** [English](README.md) | **Español**
1717
@@ -33,7 +33,7 @@
3333
<summary><strong>¿Por qué este servidor y no otro?</strong></summary>
3434

3535
- ✅ **Single-file `index.js` (2250 líneas)** — sin build, sin compilar, auditable en 1 archivo. Shebang `#!/usr/bin/env node`, listo para `node`, `yarn start` o `npx`.
36-
- ✅ **43 herramientas con JSON Schema estricto** (`additionalProperties:false`) + `try/catch` global. Cada tool retorna `content: [{type:"text"}]` y `isError:true` en fallos — nada de `// TODO`.
36+
- ✅ **47 herramientas con JSON Schema estricto** (`additionalProperties:false`) + `try/catch` global. Cada tool retorna `content: [{type:"text"}]` y `isError:true` en fallos — nada de `// TODO`.
3737
- ✅ **Cobertura Apple total:** `xcodebuild`, `simctl` (9), `devicectl` (2), `xctrace` (5 templates), `agvtool`, `security`, `osascript/xed`.
3838
- ✅ **DX moderna:** Yarn 4 vendorizado (`.yarn/releases`), `Makefile` con `help` autodocumentado, `docs/` modular, CI macOS + `make test` smoke.
3939
- ✅ **Multi-cliente:** mismo `index.js` funciona en **OpenCode**, **Codex** y **Claude Code** sin cambios.
@@ -57,6 +57,7 @@
5757
| **Assets** | 6 | `asset_list_contents`, `asset_manage_color` (Light/Dark), `asset_manage_image` (1x/2x/3x/vector), `asset_read_info`, `asset_delete`, `asset_validate_actool` (`actool`) |
5858
| **AppIcon** | 1 | `asset_generate_appicon` (TODOS los OS: iOS, macOS, watchOS, tvOS, visionOS + `sips`) |
5959
| **Paquetes / SPM** | 11 | `package_resolve`, `package_update`, `package_list_dependencies`, `package_read_resolved`, `package_reset_cache`, `package_compute_checksum`, `spm_add_dependency`, `spm_remove_dependency`, `cocoapods_manage`, `carthage_manage`, `cocoapods_to_spm_migrate` |
60+
| **Visión/UI** | 4 | `simctl_get_screen_analysis` (sips + `imagePath`), `simctl_inspect_ui_tree` (árbol accesibilidad), `simctl_tap_by_text` (tap inteligente), `simctl_fill_field` (type inteligente) |
6061

6162
---
6263

@@ -183,7 +184,7 @@ make lint
183184

184185
make test
185186
# ➜ smoke test MCP...
186-
# ✓ tools/list: 43 herramientas
187+
# ✓ tools/list: 47 herramientas
187188
# ✓ xcode_sync_strings OK
188189
# ✓ xcode_certificates_check OK
189190
# ✓ smoke test PASSED
@@ -220,10 +221,10 @@ Reinicia OpenCode / Codex / Claude Code y escribe:
220221
lista las herramientas de xcode
221222
```
222223

223-
Debes ver **43 herramientas** y en el log:
224+
Debes ver **47 herramientas** y en el log:
224225

225226
```
226-
✅ Xcode MCP Server iniciado (stdio) — 43 herramientas registradas
227+
✅ Xcode MCP Server iniciado (stdio) — 47 herramientas registradas
227228
```
228229

229230
¡Listo! Ya puedes decir:
@@ -290,7 +291,7 @@ args = ["/Users/YanxReal/Dev/Tools/Xcode-MPC/index.js"]
290291
claude mcp add xcode -- node /Users/YanxReal/Dev/Tools/Xcode-MPC/index.js
291292
# verifica
292293
claude mcp list
293-
# xcode: connected — 43 tools
294+
# xcode: connected — 47 tools
294295
```
295296

296297
O por proyecto con `.mcp.json`:
@@ -366,6 +367,10 @@ O por proyecto con `.mcp.json`:
366367

367368
`package_resolve`/`update`/`list`/`read_resolved`/`reset_cache`/`compute_checksum`, `spm_add/remove_dependency`, `cocoapods_manage`, `carthage_manage`, `cocoapods_to_spm_migrate` (Podfile→Package.swift)
368369

370+
### 12. Visión / UI Inteligente (4)
371+
372+
`simctl_get_screen_analysis` (sips → `imagePath` + `resolution` para Vision LLM), `simctl_inspect_ui_tree` (árbol accesibilidad con `center`), `simctl_tap_by_text` (`Exact`+`partial` + click centro), `simctl_fill_field` (`clearFirst` + `keystroke`)
373+
369374
> Referencia completa con JSON Schema + ejemplos copiables → [`docs/es/tools.md`](docs/es/tools.md)
370375
371376
---
@@ -378,7 +383,7 @@ make install # yarn install + chmod +x
378383
make reinstall # clean + install (desde cero)
379384
make lint # node --check index.js
380385
make doctor # Verifica Node/Yarn/Xcode/simctl/swiftlint/osascript
381-
make test # Smoke test MCP (43 tools + 2 calls)
386+
make test # Smoke test MCP (47 tools + 2 calls)
382387
make start # yarn start (stdio)
383388
make dev # yarn dev (--watch)
384389
make inspect # Inspector MCP en http://localhost:6274
@@ -396,7 +401,7 @@ Detalles → [`docs/es/development.md`](docs/es/development.md)
396401
| Doc | Para quién | Qué cubre |
397402
|---|---|---|
398403
| [`installation.md`](docs/es/installation.md) | Todos | Yarn Berry, Corepack, `yarnPath` vendorizado, troubleshooting |
399-
| [`tools.md`](docs/es/tools.md) | LLM / Dev | Las 43 tools, JSON Schema, ejemplos JSON listos para copiar |
404+
| [`tools.md`](docs/es/tools.md) | LLM / Dev | Las 47 tools, JSON Schema, ejemplos JSON listos para copiar |
400405
| [`opencode.md`](docs/es/opencode.md) | OpenCode | `opencode.json` global/local, prompts, env `DEVELOPER_DIR` |
401406
| [`codex.md`](docs/es/codex.md) | Codex | `config.toml` (`mcp_servers.xcode`), `codex mcp list` |
402407
| [`claude-code.md`](docs/es/claude-code.md) | Claude Code | `claude mcp add` / `.mcp.json`, permisos, trust |
@@ -410,8 +415,8 @@ Detalles → [`docs/es/development.md`](docs/es/development.md)
410415
```bash
411416
# Sin Make:
412417
python3 scripts/smoke_test.py
413-
# STDERR: ✅ Xcode MCP Server iniciado — 43 herramientas
414-
# ✓ tools/list: 43 herramientas
418+
# STDERR: ✅ Xcode MCP Server iniciado — 47 herramientas
419+
# ✓ tools/list: 47 herramientas
415420
# ✓ xcode_sync_strings OK
416421
# ✓ smoke test PASSED
417422

‎README.md‎

Lines changed: 15 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
[![Yarn 4](https://img.shields.io/badge/yarn-4.18-2C8EBB?logo=yarn&logoColor=white)](https://yarnpkg.com)
1212
[![MCP](https://img.shields.io/badge/MCP-Stdio_Transport-7B68EE)](https://modelcontextprotocol.io)
1313
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
14-
[![Version](https://img.shields.io/badge/version-1.2.0-blue.svg)](package.json)
14+
[![Version](https://img.shields.io/badge/version-1.3.0-blue.svg)](package.json)
1515

1616
> 🌐 **Language:** **English** | [Español](README.es.md)
1717
@@ -33,7 +33,7 @@
3333
<summary><strong>Why this server and not another?</strong></summary>
3434

3535
- ✅ **Single-file `index.js` (2250 lines)** — no build step, no compile, auditable in one file. Shebang `#!/usr/bin/env node`, ready for `node`, `yarn start` or `npx`.
36-
- ✅ **43 tools with strict JSON Schema** (`additionalProperties:false`) + global `try/catch`. Each tool returns `content: [{type:"text"}]` and `isError:true` on failure — no `// TODO`.
36+
- ✅ **47 tools with strict JSON Schema** (`additionalProperties:false`) + global `try/catch`. Each tool returns `content: [{type:"text"}]` and `isError:true` on failure — no `// TODO`.
3737
- ✅ **Full Apple coverage:** `xcodebuild`, `simctl` (9), `devicectl` (2), `xctrace` (5 templates), `agvtool`, `security`, `osascript/xed`.
3838
- ✅ **Modern DX:** Vendored Yarn 4 (`.yarn/releases`), self-documenting `Makefile` with `help`, modular `docs/`, macOS CI + `make test` smoke.
3939
- ✅ **Multi-client:** same `index.js` works with **OpenCode**, **Codex** and **Claude Code** without changes.
@@ -57,6 +57,7 @@
5757
| **Assets** | 6 | `asset_list_contents`, `asset_manage_color` (Light/Dark), `asset_manage_image` (1x/2x/3x/vector), `asset_read_info`, `asset_delete`, `asset_validate_actool` (`actool`) |
5858
| **AppIcon** | 1 | `asset_generate_appicon` (ALL Apple OS: iOS, macOS, watchOS, tvOS, visionOS + `sips` resize) |
5959
| **Package / SPM** | 11 | `package_resolve`, `package_update`, `package_list_dependencies`, `package_read_resolved`, `package_reset_cache`, `package_compute_checksum`, `spm_add_dependency`, `spm_remove_dependency`, `cocoapods_manage`, `carthage_manage`, `cocoapods_to_spm_migrate` |
60+
| **Vision/UI** | 4 | `simctl_get_screen_analysis` (sips + `imagePath`), `simctl_inspect_ui_tree` (accessibility tree), `simctl_tap_by_text` (smart tap), `simctl_fill_field` (smart type) |
6061

6162
---
6263

@@ -183,7 +184,7 @@ make lint
183184

184185
make test
185186
# ➜ smoke test MCP...
186-
# ✓ tools/list: 43 tools
187+
# ✓ tools/list: 47 tools
187188
# ✓ xcode_sync_strings OK
188189
# ✓ xcode_certificates_check OK
189190
# ✓ smoke test PASSED
@@ -220,10 +221,10 @@ Restart OpenCode / Codex / Claude Code and type:
220221
list the xcode tools
221222
```
222223

223-
You should see **43 tools** and in the log:
224+
You should see **47 tools** and in the log:
224225

225226
```
226-
✅ Xcode MCP Server started (stdio) — 43 tools registered
227+
✅ Xcode MCP Server started (stdio) — 47 tools registered
227228
```
228229

229230
Done! Now you can say:
@@ -290,7 +291,7 @@ args = ["/Users/YanxReal/Dev/Tools/Xcode-MPC/index.js"]
290291
claude mcp add xcode -- node /Users/YanxReal/Dev/Tools/Xcode-MPC/index.js
291292
# verify
292293
claude mcp list
293-
# xcode: connected — 43 tools
294+
# xcode: connected — 47 tools
294295
```
295296

296297
Or per-project with `.mcp.json`:
@@ -366,6 +367,10 @@ Or per-project with `.mcp.json`:
366367

367368
`package_resolve`/`update`/`list`/`read_resolved`/`reset_cache`/`compute_checksum`, `spm_add/remove_dependency`, `cocoapods_manage`, `carthage_manage`, `cocoapods_to_spm_migrate` (Podfile→Package.swift)
368369

370+
### 12. Vision / Smart UI (4)
371+
372+
`simctl_get_screen_analysis` (sips → `imagePath` + `resolution` for Vision LLM), `simctl_inspect_ui_tree` (accessibility tree with `center`), `simctl_tap_by_text` (`exact`/`partial` + center click), `simctl_fill_field` (`clearFirst` + `keystroke`)
373+
369374
> Full reference with JSON Schema + copy-paste examples → [`docs/tools.md`](docs/tools.md)
370375
371376
---
@@ -378,7 +383,7 @@ make install # yarn install + chmod +x
378383
make reinstall # clean + install (from scratch)
379384
make lint # node --check index.js
380385
make doctor # Check Node/Yarn/Xcode/simctl/swiftlint/osascript
381-
make test # Smoke test MCP (43 tools + 2 calls)
386+
make test # Smoke test MCP (47 tools + 2 calls)
382387
make start # yarn start (stdio)
383388
make dev # yarn dev (--watch)
384389
make inspect # MCP Inspector at http://localhost:6274
@@ -396,7 +401,7 @@ Details → [`docs/development.md`](docs/development.md)
396401
| Doc | Audience | Covers |
397402
|---|---|---|
398403
| [`installation.md`](docs/installation.md) | Everyone | Yarn Berry, Corepack, `yarnPath` vendored, troubleshooting |
399-
| [`tools.md`](docs/tools.md) | LLM / Dev | All 43 tools, JSON Schema, copy-paste JSON examples |
404+
| [`tools.md`](docs/tools.md) | LLM / Dev | All 47 tools, JSON Schema, copy-paste JSON examples |
400405
| [`opencode.md`](docs/opencode.md) | OpenCode | `opencode.json` global/local, prompts, `DEVELOPER_DIR` env |
401406
| [`codex.md`](docs/codex.md) | Codex | `config.toml` (`mcp_servers.xcode`), `codex mcp list` |
402407
| [`claude-code.md`](docs/claude-code.md) | Claude Code | `claude mcp add` / `.mcp.json`, permissions, trust |
@@ -410,8 +415,8 @@ Details → [`docs/development.md`](docs/development.md)
410415
```bash
411416
# Without Make:
412417
python3 scripts/smoke_test.py
413-
# STDERR: ✅ Xcode MCP Server started — 43 tools
414-
# ✓ tools/list: 43 tools
418+
# STDERR: ✅ Xcode MCP Server started — 47 tools
419+
# ✓ tools/list: 47 tools
415420
# ✓ xcode_sync_strings OK
416421
# ✓ smoke test PASSED
417422

‎docs/architecture.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,8 @@
1010
index.js
1111
├── Shebang + Imports (MCP SDK, child_process, fs, path, os)
1212
├── Helpers (shellEscape, expandTilde, runCommand, formatResult, findLatestXcresult)
13-
├── TOOLS (25 JSON Schema definitions)
14-
├── Handlers (25 async functions handle_*)
13+
├── TOOLS (47 JSON Schema definitions)
14+
├── Handlers (47 async functions handle_*)
1515
├── Dispatcher (HANDLERS map)
1616
└── Server (Server + StdioServerTransport + ListTools/CallTool)
1717
```

‎docs/es/architecture.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,8 @@
1010
index.js
1111
├── Shebang + Imports (MCP SDK, child_process, fs, path, os)
1212
├── Helpers (shellEscape, expandTilde, runCommand, formatResult, findLatestXcresult)
13-
├── TOOLS (25 definiciones JSON Schema)
14-
├── Handlers (25 async functions handle_*)
13+
├── TOOLS (47 definiciones JSON Schema)
14+
├── Handlers (47 async functions handle_*)
1515
├── Dispatcher (HANDLERS map)
1616
└── Server (Server + StdioServerTransport + ListTools/CallTool)
1717
```

‎docs/es/tools.md‎

Lines changed: 88 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Referencia de Herramientas (43)
1+
# Referencia de Herramientas (47)
22

33
> 🌐 **Idioma:** [English](../tools.md) | **Español**
44
@@ -278,6 +278,93 @@ Borrado seguro: valida extensión `*.colorset|*.imageset|...` y que esté dentro
278278

279279
---
280280

281+
## 10. AppIcon `Assets.xcassets` + `sips` (1 herramienta)
282+
283+
### `asset_generate_appicon`
284+
```json
285+
{ "xcassetsPath": "/ruta/a/Assets.xcassets", "iconName": "AppIcon", "baseImagePath": "/tmp/1024.png", "includeIos": true, "includeMacOs": true, "includeWatchOs": true, "includeTvOs": true, "includeVisionOs": true }
286+
```
287+
Genera `${iconName}.appiconset/Contents.json` con **42 slots** para **TODOS los OS**: iOS (iphone/ipad/ios-marketing), macOS (16-512), watchOS (38/42/45mm), tvOS (400x240/1280x768), visionOS (32/1024). Si `baseImagePath`, ejecuta `sips -z {pixels} {base} --out {slot}` por slot.
288+
289+
---
290+
291+
## 11. Paquetes / SPM / CocoaPods / Carthage (11 herramientas)
292+
293+
### `package_resolve` / `package_update`
294+
```json
295+
{ "projectPath": "/ruta/a/MyApp.xcodeproj" }
296+
```
297+
Autodetecta `Package.swift` (`swift package resolve/update`) vs `xcodebuild -resolvePackageDependencies`.
298+
299+
### `package_list_dependencies`
300+
```json
301+
{ "packageDirectory": "/ruta/a/Package.swift/dir" }
302+
```
303+
`swift package show-dependencies --format json`
304+
305+
### `package_read_resolved`
306+
```json
307+
{ "resolvedFilePath": "/ruta/a/Package.resolved" }
308+
```
309+
Soporta v1/v2/v3, retorna `{version, totalDependencies, pins}`
310+
311+
### `package_reset_cache`
312+
```json
313+
{ "projectPath": "/ruta/a/MyApp.xcodeproj" }
314+
```
315+
`xcodebuild -resetPackageCaches` + `rm -rf ~/Library/Caches/org.swift.swiftpm`
316+
317+
### `package_compute_checksum`
318+
```json
319+
{ "zipPath": "/ruta/a/MyXCFramework.zip" }
320+
```
321+
`swift package compute-checksum`
322+
323+
### `spm_add_dependency` / `spm_remove_dependency`
324+
```json
325+
{ "packageSwiftPath": "/ruta/a/Package.swift", "url": "https://github.com/Alamofire/Alamofire.git", "requirement": "from: \"5.8.0\"" }
326+
```
327+
328+
### `cocoapods_manage` / `carthage_manage`
329+
```json
330+
{ "projectPath": "/ruta/a/Podfile/dir", "action": "install" }
331+
```
332+
333+
### `cocoapods_to_spm_migrate`
334+
```json
335+
{ "podfilePath": "/ruta/a/Podfile", "packageSwiftPath": "/ruta/a/Package.swift", "dryRun": true }
336+
```
337+
338+
---
339+
340+
## 12. Visión / UI Inteligente (4 herramientas)
341+
342+
### `simctl_get_screen_analysis`
343+
```json
344+
{ "udid": "booted", "outputPath": "/tmp/sim_screen_latest.png" }
345+
```
346+
`xcrun simctl io screenshot` + `sips -g pixelWidth/Height` → `{imagePath, resolution, instructions}` para Vision LLM.
347+
348+
### `simctl_inspect_ui_tree`
349+
```json
350+
{ "udid": "booted" }
351+
```
352+
AppleScript vía archivo temporal → JSON `[{role,name,title,value,center}]`.
353+
354+
### `simctl_tap_by_text`
355+
```json
356+
{ "text": "Continuar", "exactMatch": false }
357+
```
358+
Búsqueda insensible a mayúsculas partial/exact, click centro, retorna `CLICKED:x,y`.
359+
360+
### `simctl_fill_field`
361+
```json
362+
{ "labelOrPlaceholder": "Correo", "textToType": "test@example.com", "clearFirst": true }
363+
```
364+
Localiza `role contains "text field"`, `click`, `clearFirst` con `keystroke "a" using command down`, luego `keystroke`.
365+
366+
---
367+
281368
## Manejo de errores
282369

283370
Todas las herramientas capturan `try/catch` global en `index.js:1195` y helpers `runCommand` en `index.js:39` retornan `{success, stdout, stderr, code}`. Salida formateada vía `formatResult` con `isError:true` si aplica.

0 commit comments

Comments
 (0)