This repository contains the core Computer Vision and Deep Learning pipeline for Guardie Posture, a real-time health-monitoring application. This module focuses on ergonomic posture classification using a hybrid multi-modal approach.
The system utilizes a sophisticated Early Fusion strategy to ensure high-accuracy posture detection:
- Feature Extraction: Uses MediaPipe to extract 3D skeletal coordinates (tabular data) from live webcam feeds.
- Multi-modal Fusion: Implements a hybrid architecture where:
- Tabular Data (Joint coordinates) is processed through a Multi-Layer Perceptron (MLP).
- Image Data is processed through a Fine-tuned ResNet18 CNN.
- Classification: Features are fused at an early stage to provide a final posture health score (Correct vs. Incorrect).
- Deep Learning: PyTorch, Torchvision
- Computer Vision: OpenCV, MediaPipe
- Data Processing: NumPy, Pandas, Scikit-learn
- Language: Python
- Developed and annotated a custom image dataset specifically for ergonomic posture monitoring.
- Extracted tabular joint features to augment image data, creating a robust training set for multi-modal learning.
data/: folder is for collecting tabular data and images .^early_fusion: Logic for the Early Fusion architecture (MLP + CNN) +model training and fine-tuning.posturepy: is a script that contain the helper functions for extracting features and distances.
Note: This repository showcases the core AI research and implementation of my graduation project. Full application integration is available upon request.