|
1 | | -# 🥊 Thermal Punch: Real-Time Motion Tracking Pipeline |
| 1 | +# 🥊 Thermal Punch: Real-Time Motion Intensity Visualisation Using Pseudocolour Mapping |
2 | 2 |
|
3 | 3 | ## Overview |
4 | | -**Thermal Punch** is a computer vision application that translates real-world physical movement into interactive digital inputs. Built entirely in Python as an advanced Image Processing project, the system utilizes a webcam feed to detect, isolate, and calculate the directional velocity of a user's punches in real-time. |
| 4 | +**Thermal Punch** is a hardware-agnostic, computer vision-based Exergame (Exercise + Gaming) that translates real-world physical movement into interactive digital inputs. Built entirely in Python for our Year 2 Semester 3 Image Processing module, the application provides a cost-effective, contactless fitness solution. |
5 | 5 |
|
6 | | -Beyond the core image processing algorithms, this project is architected with a strict adherence to the Software Development Life Cycle (SDLC) and features a dedicated Quality Assurance pipeline to ensure accurate, artifact-free detection. |
| 6 | +By utilizing a standard laptop webcam, the system calculates the intensity and velocity of a user's shadow-boxing movements in real-time, overlaying a dynamic "thermal" heatmap directly onto the video feed to provide immediate, gamified visual feedback on physical exertion. |
7 | 7 |
|
8 | | -## ⚙️ System Architecture & Computer Vision |
9 | | -The pipeline processes live video frames through a multi-stage filtering system: |
10 | | -* **Background Subtraction:** Utilizes OpenCV's Gaussian Mixture-Based Background/Foreground Segmentation (`cv2.createBackgroundSubtractorMOG2`) to isolate the user from static room environments. |
11 | | -* **Motion Tracking:** Implements Dense Optical Flow (Farneback algorithm) to calculate the exact magnitude and direction of the isolated movement. |
12 | | -* **State Management:** The backend logic is structured using Object-Oriented Programming (OOP), specifically leveraging the **State Pattern** to seamlessly transition the game loop between "Idle," "Attack," and "Cooldown" phases without frame-drop. |
| 8 | +--- |
| 9 | + |
| 10 | +## ⚙️ Computer Vision Techniques & Applications |
| 11 | +Rather than relying on computationally heavy AI skeletal tracking or expensive VR hardware, this project utilizes a hyper-efficient, non-learning-based "Dual-Track" computer vision architecture. |
| 12 | + |
| 13 | +* **Gaussian Mixture-Based Background Subtraction (MOG2)** |
| 14 | + * **Application:** Used to isolate the player from the static room environment. By maintaining a 500-frame historical memory, it generates a binary motion mask that effectively ignores static background objects and shadows to ensure accurate hit-detection. |
| 15 | +* **Morphological Filtering (Opening)** |
| 16 | + * **Application:** Applied immediately after background subtraction to clean the binary mask. By eroding and dilating pixels, it removes "salt-and-pepper" noise and artifacts caused by webcam grain or lighting shifts, preventing false-positive punches. |
| 17 | +* **Temporal Frame Differencing (`cv2.absdiff`)** |
| 18 | + * **Application:** Calculates the absolute pixel difference between consecutive grayscale frames. This extracts raw motion energy and acceleration, allowing the system to distinguish between a weak, slow movement and a high-intensity, intentional punch. |
| 19 | +* **Pseudocolour Mapping (JET Colourmap)** |
| 20 | + * **Application:** Transforms the raw, invisible motion intensity data into a vibrant visual spectrum. Areas with zero motion render as deep blue, while high-velocity punches render as bright red. This provides intuitive, real-time thermal-style feedback to the player. |
| 21 | +* **Dense Optical Flow (Farneback)** |
| 22 | + * **Application:** Computes the directional displacement of pixels between frames. The system uses a pixel-voting heuristic to classify the trajectory of the player's movement (Up, Left, Right) to validate directional target hits. |
| 23 | + |
| 24 | +--- |
| 25 | + |
| 26 | +## 🎮 Game Modes |
| 27 | + |
| 28 | +### 👥 Multiplayer Mode |
| 29 | +A split-screen competitive mode where Player 1 (Left) and Player 2 (Right) face off. Players must physically punch directional targets that appear on their side of the screen. Damage is dynamically scaled based on the physical intensity (speed/energy) and accuracy of the punch. |
| 30 | + |
| 31 | +### 🤖 Singleplayer Boss Fights |
| 32 | +A survival mode where Player 1 faces off against adaptive AI bosses. The game dynamically adjusts its speed based on the player's overall motion output. Each boss requires a different physical strategy to defeat: |
| 33 | + |
| 34 | +* 🔴 **MECHA-LASER** |
| 35 | + * **Ability:** Analyzes the screen to find the zone with the highest motion density and fires a devastating laser column. |
| 36 | + * **How to Beat:** Watch for the orange "DANGER!" warning box. You must physically move or lean out of that specific vertical zone before the laser fires to dodge the attack, then counter-attack. |
| 37 | +* 👁️ **THERMAL EYE (Scanner Boss)** |
| 38 | + * **Ability:** Randomly initiates a system-wide yellow scan, culminating in a "FREEZE!" command. |
| 39 | + * **How to Beat:** When the screen flashes and instructs you not to move, you must hold completely still. The boss measures raw motion strength; slight twitches trigger warnings, but high movement will result in massive damage. |
| 40 | +* 💣 **DEFLECTOR** |
| 41 | + * **Ability:** Spawns explosive thermal fireballs randomly across the screen with a ticking expiration timer. |
| 42 | + * **How to Beat:** You must physically punch the fireball's location to deflect it back and damage the boss. However, if your stamina is overheated from swinging too wildly, you will be "EXHAUSTED" and take damage instead. |
13 | 43 |
|
14 | | -## 🧪 Quality Assurance & Automated Testing |
15 | | -As an engineering priority, this system relies heavily on automated testing to prevent "ghost touches" and false-positive punch validations (e.g., pulling a hand back triggering a forward punch). |
16 | | -* **Automated CI/CD:** A comprehensive `pytest` suite is integrated to test the mathematical thresholds of the tracking modules. |
17 | | -* **Threshold Tuning:** Detection logic is fortified by a "Supermajority Floor," ensuring noise artifacts do not trigger false game states. |
18 | | -* **SDLC Tracking:** The entire development lifecycle, from bug tracking to feature implementation, is managed via an Agile Kanban board. |
| 44 | +--- |
| 45 | + |
| 46 | +## 🏆 Project Results & Objectives Achieved |
| 47 | +Based on our experimental analysis and system telemetry, the project successfully achieved its core objectives: |
| 48 | +1. **Hyper-Efficient Real-Time Tracking:** The dual-track pipeline achieved a mean latency of **2.51 ms** per frame, operating 13 times faster than the 33.3 ms requirement for 30 FPS, ensuring zero lag on standard hardware. |
| 49 | +2. **Accurate Visual Feedback:** Achieved a perfect negative correlation ($r = -1.0$) between motion intensity and Hue values, validating that the JET pseudocolour mapping is strictly monotonic and mathematically accurate. |
| 50 | +3. **Gamified Physical Exertion:** Telemetry from 773 registered interactions proved the system successfully filtered ambient room noise (threshold 25) while demanding a high average physical exertion (intensity 114.97) from the player, proving its viability as a fitness application. |
19 | 51 |
|
20 | | -📊 **[View the Live QA & Development Project Board Here](https://github.com/users/TingRongYou/projects/1/views/1)** |
| 52 | +--- |
21 | 53 |
|
22 | | -## 🚀 Getting Started |
| 54 | +## 📦 Download & Play (Releases) |
| 55 | +You do not need to install Python or any code dependencies to play the game! We have compiled the entire project into a standalone, ready-to-play executable. |
| 56 | + |
| 57 | +1. Navigate to the **[Releases](../../releases)** section on the right side of this GitHub page. |
| 58 | +2. Download the `Thermal Punch v1.0.zip` file. |
| 59 | +3. Extract the folder to your Desktop. |
| 60 | +4. Double-click **`Thermal Punch.exe`** (look for the blue boxing glove icon) to start playing! |
| 61 | + |
| 62 | +--- |
| 63 | + |
| 64 | +## 🛠️ For Developers (Source Code) |
| 65 | +If you wish to run the game from the source code or modify the computer vision pipeline: |
23 | 66 |
|
24 | 67 | ### Prerequisites |
25 | 68 | * Python 3.8+ |
26 | 69 | * A working webcam |
27 | | -* Git |
28 | 70 |
|
29 | 71 | ### Installation |
30 | | -1. Clone the repository to your local machine: |
| 72 | +1. Clone the repository: |
31 | 73 | ```bash |
32 | | - git clone https://github.com/TingRongYou/Y2S3ImageProcessing.git |
| 74 | + git clone [https://github.com/TingRongYou/Y2S3ImageProcessing.git](https://github.com/TingRongYou/Y2S3ImageProcessing.git) |
| 75 | + cd Y2S3ImageProcessing |
33 | 76 | ``` |
34 | | -2. Navigate to the project directory: |
| 77 | +2. Install the required dependencies: |
35 | 78 | ```bash |
36 | | - cd Thermal_Punch |
| 79 | + pip install opencv-python numpy pygame |
37 | 80 | ``` |
38 | | -3. Install the required dependencies: |
| 81 | +3. Run the Application: |
39 | 82 | ```bash |
40 | | - pip install opencv-python numpy pygame pytest |
| 83 | + python main.py |
41 | 84 | ``` |
42 | 85 |
|
43 | | -### Running the Application |
44 | | -To start the live computer vision tracking application: |
45 | | -```bash |
46 | | -python main.py |
47 | | -``` |
48 | | - |
49 | | -### Running the Test Suite |
50 | | -To execute the automated Quality Assurance tests: |
51 | | -```bash |
52 | | -pytest tests/ |
53 | | -``` |
54 | | - |
55 | 86 | --- |
56 | 87 |
|
57 | 88 | ## 👨💻 Author |
58 | 89 | **Ting Rong You & Yong Chong Xin & Tan Hong You** *| Final-Year Software Engineering (Honours) Student @ TARUMT* |
59 | | -* Aspiring QA Engineer & UI/UX Designer |
60 | 90 | * **LinkedIn:** [Connect with me here](https://linkedin.com/in/ting-rong-you-945aab3b6) |
0 commit comments