Skip to content

Commit 2e3d5d2

Browse files
committed
Auto Update
1 parent d874cdf commit 2e3d5d2

9 files changed

Lines changed: 206 additions & 108 deletions

File tree

docs/PHILOSOPHY.md

Lines changed: 15 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,27 @@
11
# The Philosophy of FastGhostMouse
22

33
> [!IMPORTANT]
4-
> **"Visual Feedback without Interference. Zero Focus Theft. Absolute Click-Through."**
4+
> **"Visual feedback without interference: zero focus theft, absolute click-through, and zero AWT lag."**
55
6-
FastGhostMouse is built on the principle that automation bots, AI models, and testing frameworks desperately need visual debugging, but traditional Java UI solutions destroy the very environment they try to automate.
6+
FastGhostMouse is built on the fundamental principle that robotics bots, autonomous AI models, and testing frameworks require visual debugging, but traditional Java UI solutions degrade or destroy the target environment.
77

88
## Core Tenets
99

10-
1. **Do Not Disturb the Host**
11-
An automation overlay must never steal focus (`WS_EX_NOACTIVATE`) and must never block input (`WS_EX_TRANSPARENT`). If a bot clicks, the click must reach the target app, not the overlay.
10+
### 1. Do Not Disturb the Host
11+
An automation overlay must never steal window focus (`WS_EX_NOACTIVATE`) and must never capture or block mouse input (`WS_EX_TRANSPARENT`). When a user or bot clicks, the hardware event must reach the underlying window, not the overlay.
1212

13-
2. **Bypass the AWT/Swing EDT**
14-
Standard Java windows (`JFrame`) are bound to the Event Dispatch Thread. Tying your bot's visual feedback to the heavy AWT pipeline guarantees micro-stutters and input lag. FastGhostMouse hooks directly into native Win32/DWM rendering to bypass Java's UI constraints entirely.
13+
### 2. Bypass the Java AWT Event Dispatch Thread (EDT)
14+
Standard Java UI components (`JFrame`, `JWindow`, `JDialog`) are bound to AWT's single-threaded EDT. Running high-frequency cursor animations on the EDT causes micro-stuttering, GC pressure, and input latency. FastGhostMouse runs its interpolation loop on a dedicated lightweight thread, driving DirectComposition native surfaces directly via `FastOverlay`.
1515

16-
3. **Visual Truth for AI**
17-
When training or debugging AI agents, seeing the *exact* intended path of the cursor is critical. FastGhostMouse interpolates natively to show smooth trajectories and state changes, giving developers an instant "window into the AI's mind."
16+
### 3. Visual Transparency for Autonomous Agents
17+
When training or debugging AI agents, observing the exact planned and smoothed trajectory is critical. FastGhostMouse interpolates motion continuously and renders state tags, giving developers immediate visual inspection of AI intentions without interfering with vision capture pipelines like `FastScreen`.
1818

19-
4. **Blueprint Consistency**
20-
As part of the **FastJava** ecosystem, FastGhostMouse adheres to a standardized architecture:
21-
* **Native Backend**: Direct C++ implementation via `FastOverlay`.
22-
* **Unified Loading**: Powered by `FastCore`.
23-
* **Premium Quality**: Built specifically for high-performance automation and autonomous agents.
19+
### 4. Ecosystem Synergy
20+
FastGhostMouse is designed to compose cleanly with the FastJava substrate:
21+
* **Render Pipeline**: Built directly on `FastOverlay` for DirectComposition transparency.
22+
* **Input Injection**: Integrates with `FastRobot` for native Win32 `SendInput` automation.
23+
* **JNI Substrate**: Native libraries resolved and loaded seamlessly via `FastCore`.
2424

2525
---
26-
**❤️ FastGhostMouse — Powering the next generation of Native Java.**
26+
27+
Part of the **FastJava** ecosystem. Clean architectures, zero EDT lag, maximum speed.

docs/REFERENCE.md

Lines changed: 84 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,88 @@
11
# FastGhostMouse Reference
22

3-
## 1. Native Overlay Engine
4-
* **WS_EX_TRANSPARENT**: The overlay window is mathematically ignored by the Windows input subsystem. Mouse clicks and keyboard events pass directly through to the underlying application.
5-
* **WS_EX_NOACTIVATE**: The overlay will never steal window focus from your active game, application, or browser, ensuring automation bots are not interrupted.
6-
* **WS_EX_LAYERED**: Enables per-pixel alpha transparency and high-performance image blending via standard Windows APIs.
7-
8-
## 2. API Capabilities
9-
* **useAsSecondaryMouse(x, y, index)**: Captures the active Windows cursor and seamlessly integrates it into a secondary, scriptable hardware cursor.
10-
* **moveTo(x, y)**: Initiates a hardware-accelerated, interpolated movement to the target coordinates.
11-
* **setSmoothing(factor)**: Adjusts the momentum and easing of the cursor. Lower values produce heavier, more human-like drag, while higher values snap instantly.
12-
* **setTextImage**: Attaches dynamic visual text or labels directly beneath the cursor (e.g., "Executing AI Action").
13-
14-
## 3. FastCore & FastOverlay Bindings
15-
* **JNI Hooks**: `FastGhostMouse` relies on the unified `FastCore` JNI loader to access `FastOverlayWindow` instances.
16-
* **Zero Swing/AWT Logic**: The overlay bypasses the Java Event Dispatch Thread (EDT) completely, preventing UI locks or stutters during complex bot logic.
17-
18-
## 4. Platform Support
19-
| Platform | Status |
20-
|----------|--------|
21-
| Windows 10/11 (x64) | ✅ Fully Supported |
22-
| Linux | 🚧 Planned (X11/Wayland hooks) |
23-
| macOS | 🚧 Planned |
3+
## 1. Native Architecture & Subsystems
4+
5+
`FastGhostMouse` provides a hardware-accelerated, transparent ghost cursor overlay for AI agent trajectory visualization, automation debugging, and bot telemetry.
6+
7+
It directly composites two native `FastOverlayWindow` surfaces via DirectComposition / DWM:
8+
1. **Cursor Window**: Renders either a custom high-DPI `BufferedImage` or the captured Windows OS hardware cursor (`captureSystemCursor()`).
9+
2. **Text Label Window**: Renders dynamic HUD status text or AI state labels directly offset next to the cursor without affecting OS hit testing.
10+
11+
Both windows enforce the following Win32 styles:
12+
* `WS_EX_TRANSPARENT`: Mathematically transparent to the Windows input subsystem. Mouse and keyboard events pass directly to windows below.
13+
* `WS_EX_NOACTIVATE`: Never receives or steals window focus, guaranteeing target application continuity.
14+
* `WS_EX_LAYERED`: Sub-millisecond alpha blending and per-pixel transparency composited directly by the Desktop Window Manager (DWM).
2415

2516
---
26-
**Part of the FastJava Ecosystem***Making the JVM faster. Small package. Maximum speed. Zero bloat. 🚀📋*
17+
18+
## 2. API Reference
19+
20+
### `FastGhostMouse`
21+
22+
| Method | Signature | Description | Native / Subsystem Call |
23+
|:---|:---|:---|:---|
24+
| `init(startX, startY)` | `void` | Spawns dual `FastOverlayWindow` handles at coordinates and starts the 120 FPS physics loop. | `FastOverlay.createWindow` |
25+
| `useAsSecondaryMouse(x, y, index)` | `void` | Spawns cursor, captures current OS hardware cursor, and displays overlay. | `captureSystemCursor()`, `show()` |
26+
| `moveTo(targetX, targetY)` | `void` | Sets target coordinates for smooth physics-based cursor interpolation. | Internal delta-time Euler integration |
27+
| `setCursorPositionImmediate(x, y)` | `void` | Instantly teleports the cursor overlay without animation delay. | `FastOverlayWindow.setPosition` |
28+
| `setSmoothing(factor)` | `void` | Configures momentum and easing responsiveness (default: `0.12f`). | Animation physics coefficient |
29+
| `setTextOffset(dx, dy)` | `void` | Configures pixel offset of the status label window relative to cursor tip. | Win32 layout translation |
30+
| `setCursorImage(img)` | `void` | Applies custom `BufferedImage` icon to the cursor overlay surface. | `FastOverlayWindow.updateImage` |
31+
| `setTextImage(img)` | `void` | Applies custom `BufferedImage` rendered label next to cursor. | `FastOverlayWindow.updateImage` |
32+
| `captureSystemCursor()` | `void` | Queries native Win32 `GetCursorInfo` and extracts current OS cursor bitmap. | `captureSystemCursorNative` (JNI) |
33+
| `setSystemCursorVisible(visible)` | `void` | Dynamically hides or restores the OS hardware cursor via Win32 `SetSystemCursor`. | `setSystemCursorVisible` (JNI) |
34+
| `show()` | `void` | Makes cursor and text overlays visible. | `FastOverlayWindow.show` |
35+
| `hide()` | `void` | Hides cursor and text overlays. | `FastOverlayWindow.hide` |
36+
| `dispose()` | `void` | Shuts down animation thread, releases overlay windows, and restores OS cursor. | Win32 handle cleanup |
37+
38+
---
39+
40+
### `FastGhostMouseBot`
41+
42+
| Method | Signature | Description | Subsystem Call |
43+
|:---|:---|:---|:---|
44+
| `softPress(x, y)` | `void` | Injects low-latency virtual left mouse button down event. | `FastRobot.virtualMousePress` |
45+
| `softDrag(x, y)` | `void` | Dispatches continuous coordinate stream during active drag. | `FastRobot.virtualMouseMove` |
46+
| `softRelease(x, y)` | `void` | Injects virtual left mouse button release and resets drag state. | `FastRobot.virtualMouseRelease` |
47+
| `isDragging()` | `boolean` | Queries active drag state. | Internal boolean state |
48+
49+
---
50+
51+
## 3. Platform Support
52+
53+
| Platform | Architecture | Status | Engine / Pipeline |
54+
|:---|:---|:---|:---|
55+
| **Windows 10 / 11** | `x64` | ✅ Fully Supported | Win32 Layered Windows + DirectComposition (`FastOverlay`) |
56+
| **Windows 10 / 11** | `arm64` | 🚧 Planned | Native Windows on ARM DirectComposition pipeline |
57+
| **Linux** | `x64` | 🚧 Planned | X11 Composite / Wayland Subsurface transparent overlay |
58+
| **macOS** | `x64` / `arm64` | 🚧 Planned | CoreGraphics / Metal Overlay Window |
59+
60+
---
61+
62+
## 4. Ecosystem Integration
63+
64+
```
65+
+-------------------------------------------------------------+
66+
| FastGhostMouse Application |
67+
+-------------------------------------------------------------+
68+
|
69+
+----------------------+----------------------+
70+
| |
71+
v v
72+
+-----------------------------+ +-----------------------------+
73+
| FastOverlay | | FastRobot |
74+
| (DirectComposition Canvas) | | (Native Input Injection) |
75+
+-----------------------------+ +-----------------------------+
76+
| |
77+
+----------------------+----------------------+
78+
|
79+
v
80+
+-------------------------------------------------------------+
81+
| FastCore |
82+
| (Native Library JNI Loader) |
83+
+-------------------------------------------------------------+
84+
```
85+
86+
---
87+
88+
Part of the **FastJava** ecosystem. Clean architectures, zero EDT lag, maximum speed.

docs/ROADMAP.md

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,24 @@
1-
# FastGhostMouse Roadmap 🗺️
1+
# FastGhostMouse Roadmap
22

3-
**Vision:** To provide the fastest possible native primitives for mouse input by aggressively bypassing bottlenecks in standard Java.
3+
**Vision:** High-performance, zero-latency ghost cursor and AI trajectory visualization for the JVM.
44

5-
## 🟢 v0.1.0: Initial Release (Current)
6-
- [x] **Core Native Engine**: Basic JNI implementation.
7-
- [x] **Blueprint Standards**: README, Reference, and Philosophy integration.
8-
- [ ] **Basic Performance Suite**: Initial benchmarks vs standard Java.
5+
## Current Milestone: v0.1.0 (Released)
6+
- [x] Standardized FastJava ecosystem architecture.
7+
- [x] DirectComposition GPU overlay integration via `FastOverlay`.
8+
- [x] Native hardware cursor capture and dynamic state labeling.
9+
- [x] Smooth physics-based cursor motion interpolation.
10+
- [x] Official JMH benchmark suite measuring throughput.
11+
- [x] Interactive window-repositioning demo (`examples/Demo`).
912

10-
## 🟡 v0.2.0: Optimization Phase
11-
- [ ] **SIMD Acceleration**: Implement AVX2/SSE4.2 paths for core loops.
12-
- [ ] **Software Prefetching**: Optimize memory access patterns.
13-
- [ ] **Alignment Enforcement**: Ensure zero-penalty memory boundaries.
13+
## Milestone: v0.2.0 (Performance & Vector Trails)
14+
- [ ] Hardware-accelerated motion trails (spline path drawing via Direct2D).
15+
- [ ] Multi-cursor support for parallel agent visualization.
16+
- [ ] Low-overhead coordinate ring buffer for telemetry recording.
1417

15-
## 🟠 v0.5.0: Platform & Logic Expansion
16-
- [ ] **ARM NEON Port**: Parity for Apple Silicon/Mobile.
17-
- [ ] **Advanced Features**: Multi-threaded paths and complex batch operations.
18-
19-
## 🔴 v1.0.0: Production Hardening
20-
- [ ] **Full Stability Audit**: Long-run stress testing.
21-
- [ ] **Enterprise Support**: NUMA-awareness and Large Pages support.
18+
## Milestone: v0.3.0 (Cross-Platform Foundations)
19+
- [ ] Linux X11 Composite & Wayland transparent subsurface prototypes.
20+
- [ ] macOS Metal-backed click-through window implementation.
2221

2322
---
24-
**Focus:** Performance is our USP. We optimize where Java stops.
23+
24+
Part of the **FastJava** ecosystem. Clean architectures, zero EDT lag, maximum speed.

examples/Benchmark/dependency-reduced-pom.xml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -69,9 +69,9 @@
6969
</dependencies>
7070
<properties>
7171
<jmh.version>1.37</jmh.version>
72-
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
73-
<maven.compiler.target>17</maven.compiler.target>
72+
<maven.compiler.target>21</maven.compiler.target>
7473
<uberjar.name>benchmarks</uberjar.name>
75-
<maven.compiler.source>17</maven.compiler.source>
74+
<maven.compiler.source>21</maven.compiler.source>
75+
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
7676
</properties>
7777
</project>

examples/Benchmark/pom.xml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,8 @@
99
<version>0.1.0</version>
1010

1111
<properties>
12-
<maven.compiler.source>17</maven.compiler.source>
13-
<maven.compiler.target>17</maven.compiler.target>
12+
<maven.compiler.source>21</maven.compiler.source>
13+
<maven.compiler.target>21</maven.compiler.target>
1414
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
1515
<jmh.version>1.37</jmh.version>
1616
<uberjar.name>benchmarks</uberjar.name>
@@ -32,12 +32,12 @@
3232
<dependency>
3333
<groupId>com.github.andrestubbe</groupId>
3434
<artifactId>FastCore</artifactId>
35-
<version>main-SNAPSHOT</version>
35+
<version>0.1.0</version>
3636
</dependency>
3737
<dependency>
3838
<groupId>com.github.andrestubbe</groupId>
3939
<artifactId>FastOverlay</artifactId>
40-
<version>main-SNAPSHOT</version>
40+
<version>0.1.0</version>
4141
</dependency>
4242
<dependency>
4343
<groupId>org.openjdk.jmh</groupId>

examples/Benchmark/src/main/java/fastghostmouse/benchmark/Benchmark.java

Lines changed: 42 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,9 @@
33
import fastghostmouse.FastGhostMouse;
44
import org.openjdk.jmh.annotations.*;
55

6+
import java.awt.Color;
7+
import java.awt.Graphics2D;
8+
import java.awt.image.BufferedImage;
69
import java.util.concurrent.TimeUnit;
710

811
@BenchmarkMode(Mode.Throughput)
@@ -14,20 +17,51 @@
1417
public class Benchmark {
1518

1619
private FastGhostMouse ghostMouse;
20+
private BufferedImage cursorImg;
21+
private float moveTarget = 100.0f;
1722

18-
@Setup
23+
@Setup(Level.Trial)
1924
public void setup() {
20-
try {
21-
ghostMouse = new FastGhostMouse();
22-
} catch (Throwable t) {
23-
ghostMouse = null;
25+
ghostMouse = new FastGhostMouse();
26+
ghostMouse.init(100, 100);
27+
28+
cursorImg = new BufferedImage(32, 32, BufferedImage.TYPE_INT_ARGB);
29+
Graphics2D g = cursorImg.createGraphics();
30+
g.setColor(new Color(255, 0, 0, 200));
31+
g.fillOval(0, 0, 32, 32);
32+
g.dispose();
33+
}
34+
35+
@TearDown(Level.Trial)
36+
public void tearDown() {
37+
if (ghostMouse != null) {
38+
ghostMouse.dispose();
2439
}
2540
}
2641

42+
@org.openjdk.jmh.annotations.Benchmark
43+
public void benchmarkMoveToTarget() {
44+
moveTarget = (moveTarget > 800.0f) ? 100.0f : moveTarget + 2.0f;
45+
ghostMouse.moveTo(moveTarget, moveTarget);
46+
}
47+
48+
@org.openjdk.jmh.annotations.Benchmark
49+
public void benchmarkSetCursorPositionImmediate() {
50+
ghostMouse.setCursorPositionImmediate(250, 250);
51+
}
52+
2753
@org.openjdk.jmh.annotations.Benchmark
2854
public void benchmarkSetTextOffset() {
29-
if (ghostMouse != null) {
30-
ghostMouse.setTextOffset(15.0f, -25.0f);
31-
}
55+
ghostMouse.setTextOffset(15.0f, -25.0f);
56+
}
57+
58+
@org.openjdk.jmh.annotations.Benchmark
59+
public void benchmarkSetSmoothing() {
60+
ghostMouse.setSmoothing(0.18f);
61+
}
62+
63+
@org.openjdk.jmh.annotations.Benchmark
64+
public void benchmarkUpdateCursorImage() {
65+
ghostMouse.setCursorImage(cursorImg);
3266
}
3367
}

run-benchmark.bat

Lines changed: 19 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,27 @@
11
@echo off
2-
setlocal
3-
chcp 65001 > nul
2+
setlocal enabledelayedexpansion
3+
chcp 65001 >nul
44
cd /d "%~dp0"
5-
set "MAVEN_OPTS=--enable-native-access=ALL-UNNAMED --sun-misc-unsafe-memory-access=allow -Dorg.slf4j.simpleLogger.defaultLogLevel=warn"
65

7-
echo ===================================================
8-
echo Building FastGhostMouse ^& JMH Benchmarks Uber-Jar
9-
echo ===================================================
6+
echo ================================================================================
7+
echo FastGhostMouse - Official JMH Performance Benchmark
8+
echo ================================================================================
9+
echo.
1010

11-
call mvn -q clean install -DskipTests 2>nul
12-
if %ERRORLEVEL% NEQ 0 (
13-
echo [ERROR] FastGhostMouse install failed!
14-
pause
15-
exit /b %ERRORLEVEL%
16-
)
11+
echo [1/3] Building Main Project (FastGhostMouse)...
12+
call mvn clean install -DskipTests -q
13+
if %ERRORLEVEL% NEQ 0 ( echo [ERROR] Main build failed. & pause & exit /b %ERRORLEVEL% )
1714

15+
powershell -NoProfile -Command "Unblock-File -Path '%USERPROFILE%\.fastcore\native\fastghostmouse\*', '%~dp0build\*', '%~dp0src\main\resources\*' -ErrorAction SilentlyContinue" >nul 2>&1
16+
17+
echo [2/3] Building Benchmark Uber-JAR...
1818
cd examples\Benchmark
19-
call mvn -q clean package 2>nul
20-
if %ERRORLEVEL% NEQ 0 (
21-
echo [ERROR] Benchmark packaging failed!
22-
pause
23-
exit /b %ERRORLEVEL%
24-
)
19+
call mvn clean package -DskipTests -q
20+
if %ERRORLEVEL% NEQ 0 ( echo [ERROR] Benchmark build failed. & cd ..\.. & pause & exit /b %ERRORLEVEL% )
21+
22+
echo [3/3] Running Official JMH Benchmarks for FastGhostMouse...
23+
java --enable-native-access=ALL-UNNAMED --sun-misc-unsafe-memory-access=allow -Djmh.ignoreLock=true "-Djava.library.path=..\..\src\main\resources\native;..\..\target\classes\native" -jar target\benchmarks.jar -f 1 -wi 2 -i 3 -tu ms -bm thrpt
2524

26-
echo ===================================================
27-
echo Running JMH Benchmarks (Throughput: ops/ms)
28-
echo ===================================================
29-
java --enable-native-access=ALL-UNNAMED --sun-misc-unsafe-memory-access=allow "-Djava.library.path=..\..\src\main\resources\native;..\..\target\classes\native;src\main\resources\native" -jar target\benchmarks.jar -f 1 -wi 2 -i 3 -tu ms -bm thrpt
25+
cd ..\..
26+
echo.
3027
pause

run-demo.bat

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
@echo off
2+
setlocal
3+
cd /d "%~dp0"
4+
5+
echo [1/3] Building FastGhostMouse...
6+
call mvn clean install -DskipTests -q
7+
if %ERRORLEVEL% NEQ 0 ( echo [ERROR] FastGhostMouse build failed. & pause & exit /b %ERRORLEVEL% )
8+
9+
powershell -NoProfile -Command "Unblock-File -Path '%USERPROFILE%\.fastcore\native\fastghostmouse\*', '%~dp0build\*', '%~dp0src\main\resources\*' -ErrorAction SilentlyContinue" >nul 2>&1
10+
11+
echo [2/3] Compiling Demo...
12+
cd examples\Demo
13+
call mvn compile dependency:build-classpath "-Dmdep.outputFile=cp.txt" "-DincludeScope=runtime" -q
14+
if %ERRORLEVEL% NEQ 0 ( echo [ERROR] Demo build failed. & cd ..\.. & pause & exit /b %ERRORLEVEL% )
15+
16+
echo [3/3] Running FastGhostMouse Demo...
17+
set /p CP=<cp.txt
18+
java --enable-native-access=ALL-UNNAMED --sun-misc-unsafe-memory-access=allow "-Djava.library.path=%~dp0build;%~dp0src\main\resources\native" -cp "target\classes;%CP%" fastghostmouse.demo.Demo
19+
20+
cd ..\..
21+
pause

run_demo.bat

Lines changed: 0 additions & 17 deletions
This file was deleted.

0 commit comments

Comments
 (0)