Skip to content

Commit 3b088b4

Browse files
lzc0228lzc0228
committed
Publish NavGuide source and device firmware
0 parents  commit 3b088b4

119 files changed

Lines changed: 22889 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.dockerignore‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
.git
2+
.env
3+
.env.*
4+
**/.env
5+
**/.env.*
6+
!.env.example
7+
!.env*.example
8+
!**/.env*.example
9+
.venv*
10+
__pycache__
11+
.pytest_cache
12+
.DS_Store
13+
.pio
14+
firmware/config.h
15+
build
16+
dist
17+
recordings
18+
output
19+
music
20+
voice
21+
web/static/models
22+
model
23+
*.pt
24+
*.onnx
25+
*.engine
26+
*.ts
27+
*.task
28+
*.avi
29+
*.mp4
30+
**/recordings
31+
**/uploads
32+
**/captures
33+
**/logs
34+
**/private
35+
**/secrets
36+
**/credentials
37+
**/*.wav
38+
**/*.mp3
39+
**/*.bin
40+
**/*.elf
41+
**/*.pem
42+
**/*.key
43+
**/credentials.json
44+
**/secrets.json
45+
**/service-account.json
46+
**/service_account.json
47+
**/*.local.json
48+
**/*.private.json
49+
**/*.mp4
50+
**/*.avi
51+
**/*.mov
52+
**/*.mkv
53+
**/*.m4a
54+
**/*.ogg
55+
**/*.flac
56+
**/*.aac
57+
**/*.jpg
58+
**/*.jpeg
59+
**/*.png
60+
**/*.webp
61+
**/*.gif
62+
**/*.bmp
63+
**/*.glb
64+
**/*.gltf
65+
**/*.obj
66+
**/*.pt
67+
**/*.pth
68+
**/*.onnx
69+
**/*.engine
70+
**/*.task

‎.env.example‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Local reference service. No model or cloud credential needed for observations.
2+
NAVGUIDE_HOST=127.0.0.1
3+
NAVGUIDE_PORT=8081
4+
NAVGUIDE_DEVICE_TOKEN=
5+
NAVGUIDE_FRONTEND=observations
6+
NAVGUIDE_UDP_ENABLED=false
7+
NAVGUIDE_UDP_HOST=127.0.0.1
8+
NAVGUIDE_UDP_PORT=12345
9+
NAVGUIDE_IMU_TIMEOUT_SECONDS=0.5
10+
NAVGUIDE_OBSERVATION_TIMEOUT_SECONDS=1.5
11+
NAVGUIDE_MAX_FRAME_BYTES=2097152
12+
NAVGUIDE_MAX_FRAME_PIXELS=12582912
13+
NAVGUIDE_SPEECH_ENABLED=false
14+
NAVGUIDE_SPEECH_COMMAND=espeak-ng
15+
NAVGUIDE_SPEECH_VOICE=cmn
16+
YOLOE_MODEL_PATH=model/yoloe-11l-seg.pt
17+
# NAVGUIDE_DEVICE=cuda:0
18+
# BLIND_PATH_MODEL=model/yolo-seg.pt
19+
# TRAFFIC_LIGHT_MODEL=model/trafficlight.pt
20+
# ITEM_SEARCH_MODEL=model/shoppingbest5.pt
21+
# HAND_TASK_PATH=model/hand_landmarker.task
22+
NAVGUIDE_CLOUD_ENABLED=false
23+
NAVGUIDE_CLOUD_URL=
24+
NAVGUIDE_CLOUD_TOKEN=
25+
DASHSCOPE_API_KEY=
26+
DASHSCOPE_COMPAT_BASE=https://dashscope.aliyuncs.com/compatible-mode/v1
27+
QWEN_OMNI_MODEL=qwen-omni-turbo
28+
QWEN_TEXT_MODEL=qwen-turbo
29+
CLOUD_TIMEOUT_SECONDS=30
30+
# Optional microphone recognition uses DashScope directly, not the gateway.
31+
NAVGUIDE_ASR_ENABLED=false
32+
ASR_MODEL=paraformer-realtime-v2
33+
DASHSCOPE_ASR_URL=wss://dashscope.aliyuncs.com/api-ws/v1/inference
34+
ASR_CONNECT_TIMEOUT_SECONDS=10
35+
ASR_MAX_SESSION_SECONDS=60
36+
# Extended navigation recording. Disabled by default.
37+
NAVGUIDE_RECORDING_ENABLED=false

‎.github/workflows/ci.yml‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
name: Validation
2+
on: [push, pull_request]
3+
permissions:
4+
contents: read
5+
jobs:
6+
python:
7+
runs-on: ubuntu-latest
8+
strategy:
9+
matrix:
10+
python: ["3.10", "3.11", "3.12"]
11+
steps:
12+
- uses: actions/checkout@v4
13+
- uses: actions/setup-python@v5
14+
with:
15+
python-version: ${{ matrix.python }}
16+
- run: python -m pip install -r requirements-dev.txt -c constraints.txt
17+
- run: python -m pytest -q
18+
- run: python scripts/package_source.py
19+
- name: Test the exported source package
20+
run: |
21+
python -m zipfile -e dist/NavGuide-source.zip /tmp/navguide-release
22+
cd /tmp/navguide-release/NavGuide
23+
python -m pytest -q
24+
- uses: actions/upload-artifact@v4
25+
if: matrix.python == '3.11'
26+
with:
27+
name: NavGuide-source
28+
path: dist/NavGuide-source.zip
29+
container:
30+
runs-on: ubuntu-latest
31+
steps:
32+
- uses: actions/checkout@v4
33+
- run: docker build -t navguide:test .
34+
- run: docker build -f deploy/Dockerfile.cloud -t navguide-cloud:test .
35+
- run: cp deploy/.env.cloud.example deploy/.env.cloud
36+
- run: docker compose -f deploy/compose.cloud.yaml config --quiet
37+
firmware:
38+
runs-on: ubuntu-latest
39+
steps:
40+
- uses: actions/checkout@v4
41+
- uses: actions/setup-python@v5
42+
with:
43+
python-version: '3.12'
44+
- run: python -m pip install platformio==6.1.18
45+
- run: bash scripts/test_firmware.sh
46+
- run: pio run -e firmware_ci -e firmware_ci_tls
47+
- uses: actions/upload-artifact@v4
48+
with:
49+
name: firmware-ci-unconfigured
50+
path: |
51+
.pio/build/firmware_ci/*.bin
52+
.pio/build/firmware_ci_tls/*.bin

‎.gitignore‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
.DS_Store
2+
__pycache__/
3+
*.py[cod]
4+
.pytest_cache/
5+
.venv*/
6+
.env
7+
.env.*
8+
!.env.example
9+
!*.env.example
10+
!.env*.example
11+
firmware/config.h
12+
.pio/
13+
build/
14+
dist/
15+
recordings/
16+
music/
17+
voice/
18+
web/static/models/
19+
output/
20+
*.log
21+
*.pt
22+
*.onnx
23+
*.engine
24+
*.ts
25+
*.task
26+
*.avi
27+
*.mp4
28+
*.wav
29+
*.mp3
30+
*.bin
31+
*.elf
32+
*.pem
33+
*.key
34+
credentials.json
35+
secrets.json
36+
service-account.json
37+
service_account.json
38+
*.local.json
39+
*.private.json
40+
model/*
41+
!model/README.md
42+
!model/manifest.example.json
43+
.coverage
44+
htmlcov/

‎CITATION.cff‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
cff-version: 1.2.0
2+
message: "Please cite the NavGuide paper when using the algorithm. Publication details are pending."
3+
title: "NavGuide: Task-Aware Visual Content Selection and Inertial Gating for Wearable Spoken Guidance"
4+
type: software
5+
authors:
6+
- family-names: Li
7+
given-names: Shicheng
8+
- family-names: Yang
9+
given-names: Fengjiao
10+
- family-names: Zhang
11+
given-names: Jiuhan
12+
- family-names: Zhang
13+
given-names: Jiaming
14+
repository-code: https://github.com/InSAI-Lab/NavGuide

‎CONTRIBUTING.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# 参与贡献
2+
3+
使用 Python 3.10 至 3.12，在虚拟环境中安装开发依赖：
4+
5+
```bash
6+
python -m pip install -r requirements-dev.txt -c constraints.txt
7+
python -m pytest -q
8+
```
9+
10+
算法变更应说明对应步骤、参数来源和对候选选择、重复抑制、运动门控的影响。修复应包含能覆盖问题的测试。设备或云端测试结果需说明硬件、依赖版本、测量时钟与输入来源。
11+
12+
固件变更运行：
13+
14+
```bash
15+
bash scripts/test_firmware.sh
16+
FIRMWARE_ENV=firmware_ci bash scripts/build_firmware.sh
17+
```
18+
19+
提交应聚焦功能、修复或必要文档。密钥、私有固件配置、个人录制、模型二进制和未授权资产不得提交。保留第三方版权与许可证。
20+
21+
算法评估的数据、参数和测量要求见 [测试与复现条件](docs/architecture.md#测试与复现条件)。

‎Dockerfile‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
FROM python:3.11-slim-bookworm
2+
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
3+
WORKDIR /app
4+
RUN apt-get update && apt-get install -y --no-install-recommends espeak-ng \
5+
&& rm -rf /var/lib/apt/lists/* \
6+
&& useradd --create-home --uid 10001 navguide
7+
COPY requirements.txt constraints.txt ./
8+
RUN pip install --no-cache-dir -r requirements.txt -c constraints.txt
9+
COPY --chown=navguide:navguide . .
10+
USER navguide
11+
EXPOSE 8081 12345/udp
12+
HEALTHCHECK --interval=30s --timeout=3s --start-period=15s --retries=3 \
13+
CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8081/api/ready', timeout=2)"
14+
CMD ["python", "-m", "navguide"]

‎LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2025 AI-FanGe
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎PROJECT_STRUCTURE.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# 项目结构
2+
3+
| 路径 | 内容 |
4+
| :--- | :--- |
5+
| `navguide/__main__.py` | 本地服务命令行入口 |
6+
| `navguide/core/context.py` | 任务、场景平滑和用户偏好 |
7+
| `navguide/core/selection.py` | 同类去重、五因子评分、Top 3 和语义签名 |
8+
| `navguide/core/gating.py` | 躯干角速度门控与紧急、目标绕过 |
9+
| `navguide/core/proximity.py` | 相机内参与类别尺寸先验的粗距离估算 |
10+
| `navguide/core/phrasing.py` | 中文、英文行动优先短句 |
11+
| `navguide/core/pipeline.py` | P、T、B 条件与分阶段统计 |
12+
| `navguide/i18n.py`, `navguide/locales/` | 中文文案、语音模板与共享关键词集合 |
13+
| `navguide/runtime/` | FastAPI 服务、环境配置、设备认证、IMU、视觉和语音 |
14+
| `navguide/cloud/` | 云描述、标签映射、语音识别和独立网关 |
15+
| `navguide/navigation/` | 导航应用、任务调度、盲道与过街工作流 |
16+
| `navguide/perception/` | 视觉检测、识别与跟踪 |
17+
| `navguide/audio/` | 音频处理、播放与命令解析 |
18+
| `navguide/io/` | 设备通信与媒体输入输出 |
19+
| `firmware/`, `platformio.ini` | ESP32 固件与构建配置 |
20+
| `deploy/` | 云容器、Compose 和 HTTPS 配置 |
21+
| `scripts/evaluate.py` | 相同检测输入下的离线回放 |
22+
| `scripts/package_source.py` | 源代码打包和凭据检查 |
23+
| `scripts/` | 固件测试、构建与模型校验 |
24+
| `examples/` | 回放输入与相机标定配置示例 |
25+
| `tests/` | 算法、协议、服务、云端和固件测试 |
26+
| `docs/` | 算法、部署、硬件与发布说明 |
27+
| `web/index.html` | 本地服务控制台 |
28+
| `web/templates/`, `web/static/` | 扩展导航页面与静态资源 |
29+
| `.github/workflows/` | Python、容器和固件持续集成 |
30+
| `model/` | 模型说明、清单模板和本地权重 |
31+
| `voice/`, `music/`, `recordings/` | 本地音频资产和录制输出 |
32+
33+
`.env`、`firmware/config.h`、模型权重、录制资料和构建产物不进入源代码包。模型与音频资源按各自许可单独分发。

‎README.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# NavGuide
2+
3+
Wearable spoken guidance with task-aware visual content selection and inertial gating. NavGuide includes a local device service, tactile paving and street crossing workflows, item search, an optional cloud description gateway, and XIAO ESP32S3 Sense firmware.
4+
5+
## Quick Start
6+
7+
Use Python 3.11:
8+
9+
```bash
10+
git clone https://github.com/InSAI-Lab/NavGuide.git
11+
cd NavGuide
12+
python3.11 -m venv .venv
13+
source .venv/bin/activate
14+
python -m pip install -r requirements-dev.txt -c constraints.txt
15+
cp .env.example .env
16+
python -m navguide
17+
```
18+
19+
Open <http://127.0.0.1:8081>. The default `observations` mode accepts detection results without loading a vision model. Camera navigation requires the models and device setup described below.
20+
21+
Run tests with `python -m pytest -q`.
22+
23+
## Documentation
24+
25+
- [Local deployment, device interfaces, and field validation](docs/deployment.md)
26+
- [Cloud gateway deployment](docs/cloud.md)
27+
- [Hardware wiring, firmware configuration, and flashing](docs/hardware.md)
28+
- [Model files and dependencies](model/README.md)
29+
- [Algorithm, parameter provenance, and evaluation](docs/architecture.md)
30+
- [Project structure](PROJECT_STRUCTURE.md)
31+
32+
For extended navigation, install `requirements-navigation.txt`, prepare the task models and audio assets, then run `python -m navguide.navigation.app`.
33+
34+
## License
35+
36+
See [LICENSE](LICENSE) and [Third-Party Notices](THIRD_PARTY_NOTICES.md) for license terms and attribution.

0 commit comments

Comments
 (0)