🐢 Multi-Agent TurtleBot3 CV Navigation Stack

This repository contains a high-performance, lightweight overhead-camera localization and direct velocity-control stack for multi-agent control of three TurtleBot3 Burger robots.

The system operates centrally without the overhead of AMCL, Nav2, map servers, planner/controller servers, or lidar-based navigation.

Note

This package requires the standard TurtleBot3 ROS 2 dependencies. For hardware configuration and environment setup, please refer to the official ROBOTIS TurtleBot3 e-Manual / ROS 2 Setup Guide.

---

🎬 Multi-Agent Control in Action

Here is the TurtleBot3 Burger fleet navigating conflicts and reaching target goals using our localization and control stack:

Episode 1 (0:02 - 0:28)

Overhead View	Side View

Episode 2 (0:39 - 1:26)

Overhead View	Side View

Episode 3 (1:35 - 2:09)

---

✨ Features

• Real-time Overhead Localization: Fuses overhead camera coordinates with local odometry yaw deltas using an Extended Kalman Filter (EKF).
• Scheduled Planner Mode: Generates conflict-free joint schedules offline using Conflict-Based Search (CBS) or Prioritized Planning, tracked by a path follower and priority-aware ORCA local velocity filter.
• Interactive Operator GUI: Facilitates easy point-and-click robot localization, initial heading setup, target assignment, and real-time path visualization.

---

🛠️ TurtleBot3 Hardware Setup (From Scratch)

If you are setting up the TurtleBot3 Burger robots from a clean state:

1. SBC Setup: Install the Ubuntu Server IoT image and ROS 2 Humble on each robot's Raspberry Pi by following the ROBOTIS TurtleBot3 Quick Start / SBC Setup Guide.
2. OpenCR Setup: Flash the standard TurtleBot3 Burger firmware onto the OpenCR micro-controller following the OpenCR Setup Guide.
3. Wi-Fi Connection: Configure each robot's network client to auto-connect to the operator workspace router SSID (netgear11).
4. Environment Variables: Configure the network env vars in the ~/.bashrc on each robot (replacing 192.168.1.XX with the robot's IP):

   export ROS_DOMAIN_ID=30
   export TURTLEBOT3_MODEL=burger
   export ROS_LOCALHOST_ONLY=0

5. Validation: Test SSH connection from your operator PC and verify that running ros2 launch turtlebot3_bringup robot.launch.py on the robot successfully publishes /odom and registers on the network.

---

⚙️ Quick Start

1. Clone & Build the Workspace

Clone this repository and compile the workspace using colcon:

git clone <repository_url>
cd ~/turtlebot_ws
source /opt/ros/humble/setup.bash
colcon build --symlink-install
source install/setup.bash

2. Connect to Wi-Fi

Important

Network Connection: Connect the operator PC to the dedicated TurtleBot3 Wi-Fi network (SSID: netgear11). Do not connect via wired Ethernet, as the robots communicate exclusively over this wireless network.

3. Remote Robot Bringup (Terminal A)

Ensure the robots and your PC share the same domain and start remote hardware bringup over SSH:

cd ~/turtlebot_ws
source install/setup.bash
export ROS_DOMAIN_ID=30
ros2 run ros_multi_robot_navigation bringup_lab_robots \
  --identity-file ~/.ssh/turtlebot_lab_ed25519

4. Run the Fleet Controller (Terminal B)

Start the central operator control node:

cd ~/turtlebot_ws
source install/setup.bash
export ROS_DOMAIN_ID=30
ros2 launch cv_localization cv_mppi_direct.launch.py

---

🎮 Operator GUI Workflow

Once the "CV Scheduled Multi-Agent Control" GUI window is visible:

1. Identify: Click on each visible robot blob (tb_1 🔴, tb_2 🔵, tb_3 🟢) to assign identities.
2. Heading: Click a second point slightly ahead of each robot's front to initialize the EKF yaw.
3. Goals: Click goal coordinates for each robot in sequential order (tb_1 → tb_2 → tb_3).
4. Run: Press Enter to start motion.

Keyboard Shortcuts

Key	Action
Enter	Commences motion control after goal setting.
Space	Emergency Stop (commands zero velocity to all robots instantly).
Esc	Emergency Stop (commands zero velocity to all robots instantly).
r	Resets and clears goals and planners.
q	Stops all robots and terminates the GUI.

---

📋 Robot Fleet Registry

Configuration file location: src/multi_robot_navigation_ROS2/config/robots.yaml

Robot	Host IP	SSH Login	ROS Domain ID
tb_1	192.168.1.20	turtlebot@192.168.1.20	30
tb_2	192.168.1.15	ubuntu@192.168.1.15	30
tb_3	192.168.1.16	ubuntu@192.168.1.16	30

---

📡 ROS Topics & Services

Subscribed & Published Topics

• /tb_N/cv_pose (Pub): Fused camera/EKF robot pose from GUI.
• /tb_N/odom (Sub): Local odometry yaw deltas for fusion EKF.
• /tb_N/mppi_goal (Pub): Position target coordinates.
• /tb_N/offline_plan (Pub): Computed multi-agent scheduled path preview.
• /tb_N/cmd_vel (Pub): Final velocity command dispatched to the robot.
• /fleet_mppi/status (Pub): Global fleet controller event logs.

Operational Services

• /fleet_mppi/start (Trigger): Begins path execution.
• /fleet_mppi/stop (Trigger): Immediately halts all robots.
• /fleet_mppi/clear_goals (Trigger): Resets goals.
• /fleet_mppi/plan (Trigger): Manually triggers offline path generation.

---

📚 Advanced References & Configuration

All system architecture, package structure details, configuration matrices, and setup tools are located in DETAILS.md:

• 📐 System Architecture: Signal flow, camera tracking pipeline, and safety loop diagram.
• 📁 Workspace Layout: Detailed outline of repository directories and internal package boundaries.
• 🧠 Algorithm Planners: Deep dive into CBS, SIPP, and ORCA.
• 🔑 SSH Passwordless Configuration: Setting up secure, automated remote bringup.
• 📷 Workspace & Homography Calibration: How to align camera pixels to physical world coordinates.
• 🖼️ Static Background Capture: Setting up static background subtraction.
• ⚙️ Planner & Control Parameters Reference: Descriptions of variables in config.yaml.
• 🛡️ Safety Envelope Guards: Trigger bounds and automatic shutdown triggers.
• 🩺 Troubleshooting & Diagnostics: Common errors, detection adjustments, and networking resolutions.