Vision-Guided UAV Navigation

A Python software stack for autonomous UAV navigation and target interaction using YOLOv8, OpenCV, Raspberry Pi, MAVLink, and ArduPilot.

The system combines GPS waypoint navigation with local visual servoing. It detects a selected target, verifies its color, estimates distance from monocular imagery, generates body-frame velocity and yaw commands, maintains altitude, and controls an onboard manipulator.

What is included

• YOLOv8 target detection
• HSV-based target-color verification
• Monocular distance estimation
• PD-based target centering
• Forward-distance and altitude regulation
• MAVLink body-frame velocity commands
• GPS waypoint navigation
• Target-search and recovery logic
• Raspberry Pi camera integration
• Manipulator servo control
• MJPEG monitoring stream
• HSV-assisted YOLO annotation utility

Architecture

flowchart LR
    Camera[Pi Camera] --> Detection[YOLOv8 detection]
    Detection --> Verification[HSV target verification]
    Verification --> Range[Distance estimation]
    Range --> Control[Visual control]
    Control --> MAVLink[MAVLink commands]
    MAVLink --> FlightController[ArduPilot / Pixhawk]
    FlightController --> Telemetry[GPS + attitude + altitude]
    Telemetry --> Mission[Mission logic]
    Mission --> Control
    Mission --> Manipulator[Servo actuation]

Loading

A longer description is available in docs/architecture.md.

Repository structure

vision-guided-uav-navigation/
├── config/
│   └── mission.example.json
├── docs/
│   └── architecture.md
├── media/
│   └── vision_tracking_demo.png
├── models/
│   └── README.md
├── scripts/
│   ├── run_full_mission.py
│   ├── run_gps_waypoint_test.py
│   ├── run_hsv_detector.py
│   ├── run_manipulator_test.py
│   ├── run_orbit_demo.py
│   ├── run_vision_bench.py
│   └── run_visual_servoing.py
├── tools/
│   └── auto_label_dataset.py
└── uav_nav/
    └── runtime.py

Project context

Developed as part of the Aerial Robotics course at the American University of Beirut.

I designed and implemented all software in this repository, including the perception pipeline, visual control, MAVLink communication, autonomous mission logic, GPS navigation, camera integration, and manipulator control.

Installation

The hardware scripts were developed for a Raspberry Pi environment with Picamera2.

python3 -m venv --system-site-packages .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -e .

On Raspberry Pi OS, install Picamera2 through the system package manager if it is not already available:

sudo apt update
sudo apt install python3-picamera2

Model weights

The trained YOLO weights are not included in the repository.

Place them at:

models/best.pt

or set:

export UAV_MODEL_PATH=/absolute/path/to/best.pt

Private mission configuration

Real GPS coordinates are deliberately excluded.

cp config/mission.example.json config/mission.json

Edit config/mission.json locally. The file is ignored by Git and should not be committed.

A different configuration path can be selected with:

export UAV_MISSION_CONFIG=/absolute/path/to/mission.json

The serial connection can also be overridden:

export UAV_CONNECTION=/dev/ttyAMA0
export UAV_BAUD=57600

Running the software

Vision-only bench test

This mode displays the estimated commands but does not send MAVLink commands:

python -m scripts.run_vision_bench

Classical HSV detector

python -m scripts.run_hsv_detector

Flight-capable scripts

Flight and actuator scripts are disabled by default. After simulation, hardware, environment, and safety checks:

export UAV_ENABLE_FLIGHT=YES

Then run one of:

python -m scripts.run_visual_servoing
python -m scripts.run_gps_waypoint_test
python -m scripts.run_orbit_demo
python -m scripts.run_manipulator_test
python -m scripts.run_full_mission

Annotation helper

python -m tools.auto_label_dataset \
    data/balloons/train/images \
    data/balloons/train/labels

The generated annotations are only coarse initial labels and must be manually checked before training.

Evaluation status

The software was tested through component-level tests, simulation, and outdoor flight experiments during the course project. The public version removes private coordinates and machine-specific paths but preserves the implemented mission logic.

The control gains, camera calibration, PWM values, flight altitudes, and safety thresholds are specific to the original experimental platform. They must not be reused on another vehicle without validation.

Safety

This repository contains experimental autonomous-flight and actuator-control code. Do not run flight-capable scripts without:

• validating commands in simulation first;
• checking the vehicle configuration, propeller clearance, failsafes, geofence, and emergency-stop procedure;
• confirming camera calibration, coordinate conventions, control signs, and servo limits;
• operating in a controlled area under applicable UAV regulations.