LiberEye coordinates speech and wrist haptics for assistive navigation. A phone relays camera or glasses frames to a perception backend, then delivers the speech and Bluetooth wrist cues selected by Event-Priority Multimodal Coordination (EPMC).
The implementation follows Algorithm 1, equations (2) and (3), and Table 1 of LiberEye: Event-Priority Coordination of Speech and Wrist Haptics for Assistive Navigation.
Supervised outdoor sessions with four blind participants using white canes. Faces are obscured.
- Backend: perception, EPMC, authenticated HTTP API and deployment configuration.
- iOS app: phone camera, cloud relay, wrist control and optional licensed glasses integration.
- Wrist firmware: nRF52840 and DRV2605L reference implementation, wiring and flashing instructions.
Use Python 3.11 or 3.12:
cd backend
python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install -c constraints.txt '.[dev,hardware]'
python scripts/configure_cloud.py
python scripts/run_cloud.pyThe server listens on 127.0.0.1:8080. The configuration script creates a private .env with an API token. See the backend guide for model and OCR setup, and the deployment guide for server, domain and app settings.
See CITATION.cff for citation metadata and third-party notices for dependency terms. No redistribution license for the project source has been specified.
@unpublished{li2026libereye,
author = {Li, Shicheng and Zhang, Jiuhan and Yang, Fengjiao and Zhang, Jiaming},
title = {{LiberEye}: Event-Priority Coordination of Speech and Wrist Haptics for Assistive Navigation},
year = {2026},
note = {Under submission to ICASSP 2027}
}
