This demonstration shows the complete industrial workflow:
- Conveyor monitoring
- AI vision inspection
- CNN classification
- Expert system decision making
- MATLAB robot controller
- Optimization-based path planning
- CoppeliaSim digital twin execution
- Arduino HMI synchronization
- OK and Defective sorting cycles
This project presents a complete Cyber-Physical Production System (CPPS) designed for automated industrial quality inspection and intelligent sorting.
The system combines Artificial Intelligence, Robotics, Optimization Algorithms, Digital Twin Technology, TCP/IP Communication, and Hardware-In-The-Loop integration to simulate a modern Industry 4.0 manufacturing cell.
A camera-based inspection system analyzes industrial casting parts moving on a conveyor. A custom Convolutional Neural Network (CNN) classifies each part as either OK or Defective.
The AI decision is translated by an Expert System into robotic actions, which are executed through MATLAB, optimized using multiple planning algorithms, and visualized in real-time inside a CoppeliaSim Digital Twin environment.
The final system performs fully automated inspection and sorting without human intervention.
- Real-Time AI Vision Inspection
- Custom CNN Defect Detection Model
- Expert System Decision Layer
- TCP/IP Distributed Architecture
- MATLAB Robotics Control Center
- Forward & Inverse Kinematics
- Multi-Algorithm Optimization Framework
- CoppeliaSim Digital Twin
- Hardware-In-The-Loop Integration
- Arduino LCD & Status Indicators
- Autonomous Conveyor-Based Sorting
- Manual and Automatic Operating Modes
- Emergency Stop Functionality
Industrial Camera
│
▼
Image Acquisition
│
▼
CNN Vision Model
│
▼
Expert System
│
▼
TCP/IP Communication
│
▼
MATLAB Controller
│
▼
Optimization Algorithms
│
▼
Inverse Kinematics
│
▼
UR5 Robot
│
▼
Pick & Place Action
│
▼
Arduino HMI Feedback
The project operates using a distributed communication framework.
+-------------------------+
| CoppeliaSim |
| Conveyor + UR5 + Sensor |
+-------------------------+
|
| Port 19000
|
▼
TCP/IP
+-------------------------+
| MATLAB |
| IK / FK / Optimizers |
+-----------+-------------+
|
| Port 65432
|
▼
+-------------------------+
| Python AI Core |
| CNN + Expert System |
+-----------+-------------+
|
| Port 19001
|
▼
+-------------------------+
| Arduino Serial Bridge |
| LCD + LEDs |
+-------------------------+
- Python
- PyTorch
- OpenCV
- TorchVision
- MATLAB
- Robotics System Toolbox
- Instrument Control Toolbox
- Global Optimization Toolbox
- CoppeliaSim (V-REP)
- Arduino UNO
- LCD Display
- LED Indicators
The dataset used in this project is not included in this repository due to size limitations. the link to download it : https://www.kaggle.com/datasets/ravirajsinh45/real-life-industrial-dataset-of-casting-product?resource=download
The AI vision model performs binary classification of industrial casting products:
- OK Parts
- Defective Parts
Defective samples include multiple defect patterns commonly found in casting processes, such as cracks, scratches, and surface imperfections.
Dataset Structure:
casting_data/
├── train/
│ ├── ok_front/
│ └── def_front/
│
└── test/
├── ok_front/
└── def_front/
Preprocessing Pipeline:
- RGB Image
- Grayscale Conversion
- Resize to 64×64
- Tensor Conversion
- CNN Inference
The inspection subsystem was developed using PyTorch and OpenCV.
Processing Pipeline:
Camera → Image Processing → CNN → Classification → Expert System
The custom CNN architecture was designed specifically for binary defect classification.
Input Image (64×64×1)
│
▼
Conv2D (1 → 16)
│
▼
ReLU
│
▼
MaxPool
│
▼
Conv2D (16 → 32)
│
▼
ReLU
│
▼
MaxPool
│
▼
Flatten
│
▼
FC (8192 → 128)
│
▼
ReLU
│
▼
FC (128 → 2)
│
▼
Output
(OK / DEFECTIVE)
Training Configuration:
- Optimizer: Adam
- Learning Rate: 0.001
- Loss Function: CrossEntropyLoss
- Binary Classification
- Real Industrial Dataset
Evaluation Metrics:
- Accuracy
- Precision
- Recall
- F1 Score
- Confusion Matrix
The Expert System converts AI predictions into deterministic industrial actions.
Decision Rules:
IF Part = OK
→ Packaging Line
→ Green Bin
→ Y = +500
IF Part = Defective
→ Scrap Bin
→ Red Bin
→ Y = -500
Generated Outputs:
- Robot Coordinates
- Sorting Decision
- Arduino Signals
- MATLAB Commands
MATLAB acts as the master controller of the entire system.
Responsibilities:
- TCP Server
- Robot Control
- Forward Kinematics
- Inverse Kinematics
- Optimization
- Benchmarking
- Digital Twin Synchronization
Implemented Path Planners:
- Analytical Inverse Kinematics
- Genetic Algorithm (GA)
- Simulated Annealing (SA)
- Particle Swarm Optimization (PSO)
- Pattern Search
Evaluation Criteria:
- Joint Motion Cost
- Energy Consumption
- Position Accuracy
- Smoothness
- Computational Time
The GUI supports:
Run ALL & Benchmark Best
for automatic comparison and optimal solution selection.
The industrial workcell was modeled inside CoppeliaSim.
Components:
- Conveyor Belt
- Proximity Sensor
- Sorting Stations
- UR5 Robot
- Pick-and-Place Workspace
The project includes a physical Arduino subsystem acting as an industrial HMI.
Features:
- LCD Display
- Status LEDs
- Real-Time Feedback
- TCP-to-Serial Integration
- Sensor Driven
- Fully Autonomous
- Continuous Production
- User Controlled
- Editable Coordinates
- Testing & Debugging
Press:
Q
to safely terminate:
- Python
- MATLAB
- Arduino
- TCP Connections
git clone https://github.com/USERNAME/Industrial-Sorting-System.gitpip install torch torchvision opencv-python matplotlib pillow pyserial scikit-learn- MATLAB R2023a or newer
- CoppeliaSim
- Arduino IDE
Update image paths inside:
train_model.py
evaluate_model.py
realtime_conveyor.py
Update COM port in:
bridge.py
if auto-detection fails.
IMPORTANT:
Run programs in the following order:
- Launch CoppeliaSim Scene
- Run bridge.py
- Open MATLAB GUI
- Click Connect
- Run realtime_conveyor.py
The system is now fully synchronized.
✔ AI-Based Defect Detection
✔ Digital Twin Implementation
✔ Hardware-In-The-Loop Integration
✔ Real-Time TCP/IP Communication
✔ Optimization-Based Motion Planning
✔ MATLAB-CoppeliaSim Synchronization
✔ Industrial Automation Workflow
✔ Industry 4.0 Concepts
This project was developed as part of the MAE401 – Artificial Intelligence course at the Faculty of Engineering, Benha University.
Dr. Amro Shafik Project Supervisor |
Eng. Mohamed Nasser Teaching Assistant |
Mahmoud Mohamed Shamekh |
Omar Mahmoud Metwally |
Omar Shokran |
Mohamed Abdeltawab |
We would like to express our sincere appreciation to our supervisors for their guidance, support, and valuable feedback throughout the development of this project.
We also thank all team members for their dedication, collaboration, and continuous efforts that contributed to the successful completion of this work.
This repository is intended for educational, research, and demonstration purposes.