You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+14-6Lines changed: 14 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,7 @@ A Set-of-Mark (SoM) detection pipeline for macOS that transforms screenshots int
6
6
7
7

8
8
9
-
*151 numbered elements detected in ~0.8s — text labels (Apple Vision), rectangles, icons, and buttons (Florence-2). [Full manifest JSON →](docs/examples/vscode-manifest.json)*
9
+
*151 numbered elements detected in ~1.7s — text labels (Apple Vision), rectangles, icons, and buttons (Florence-2). [Full manifest JSON →](docs/examples/vscode-manifest.json)*
End-to-end on a 1920x1080 VS Code screenshot (~151 UI elements detected):
51
-
-**~0.8s** with fast OCR (Florence-2 ~650ms + Vision ~189ms)
52
-
-**~1.6s** with accurate OCR (Florence-2 ~650ms + Vision ~980ms)
51
+
-**~1.7s** with fast OCR (Florence-2 ~1.5s + Vision ~213ms)
52
+
-**~2.6s** with accurate OCR (Florence-2 ~1.5s + Vision ~977ms)
53
53
54
54
## Quick Start
55
55
@@ -112,6 +112,14 @@ Options:
112
112
--backend BACKEND Detection backend: auto (default), coreml, mlx
113
113
```
114
114
115
+
## Documentation
116
+
117
+
-[API Reference](docs/api.md) — Functions, types, and manifest schema
118
+
-[Performance](docs/performance.md) — Benchmarks and optimization tips
119
+
-[Troubleshooting](docs/troubleshooting.md) — Common issues and FAQ
120
+
-[Research Background](docs/research.md) — Model selection and benchmark methodology
121
+
-[Contributing](CONTRIBUTING.md) — Setup and PR guidelines
122
+
115
123
## Requirements
116
124
117
125
-**macOS** (Apple Vision Framework is macOS-only)
@@ -123,7 +131,7 @@ Options:
123
131
124
132
uitag supports pluggable detection backends via the `DetectionBackend` protocol:
125
133
126
-
-**MLX** (default) — Florence-2 inference on GPU via Metal. ~160ms per quadrant on M2 Max.
134
+
-**MLX** (default) — Florence-2 inference on GPU via Metal. ~220ms per quadrant on M2 Max.
127
135
-**CoreML** — DaViT vision encoder on Apple Neural Engine, decoder on GPU. Useful when GPU is contended by other workloads. Requires a converted model (`python tools/convert_davit_coreml.py`).
0 commit comments