Conveyor Belt Damage Detection

Ripik AI — Take Home Assignment

---

Project Structure

.
├── pipeline.py                   # Main inference script (primary deliverable)
├── train.ipynb                   # Dataset analysis + threshold calibration (Jupyter Notebook)
├── eval.py                       # Metrics evaluation script
├── classifier_train.py           # EfficientNet-B0 classifier (auxiliary — evaluation only)
├── README.md                     # This file
├── requirements.txt              # Python dependencies
│
├── detections_summary.xlsx       # All 359 image detections compiled into Excel
├── manual_review_SAMPLE.xlsx     # 136 manually reviewed images (human ground truth proxy)
├── metadata.csv                  # Belt polygon metadata extracted from label files
├── predictions_summary.csv       # CNN predictions for remaining 223 unlabelled images
├── metrics_report.json           # Evaluation metrics output (Precision/Recall/F1/mF1)
│
├── classifier/                   # Trained EfficientNet-B0 weights
│   └── best_model.pth
│
├── outputs_v3/                   # FINAL pipeline outputs — all 359 images (V3)
│   ├── <image_name>.jpg          # Annotated image — red=scratch, green=edge_damage
│   └── <image_name>.json         # Detection JSON in required format
│
├── outputs/                      # Initial pipeline outputs (V1 baseline run)
│   ├── <image_name>.jpg
│   └── <image_name>.json
│
├── images/                       # Input images (provided onsite)
│   └── <image_name>.jpg
│
└── labels/                       # Input YOLO polygon labels (provided onsite)
    └── <image_name>.txt

---

Approach

This solution uses Classical Computer Vision — no deep learning for the main detection pipeline.

Why Classical CV?

The dataset contains only belt-outline segmentation labels (class 0 polygons). No damage bounding-box ground truth exists. Supervised CNN/YOLO training is not feasible without labelled damage instances.

Detection Pipeline

Image
  │
  ├─► Parse belt polygon (.txt) → belt mask → erode 20px inward
  │
  ├─► CLAHE normalisation (day/night auto-detected via brightness)
  │
  ├─► Residual map = |image − GaussianBlur(image, 61×61)|
  │
  ├─► SCRATCH DETECTION
  │     • Tiered adaptive threshold (belt_median < 3 → pct=97, else pct=85)
  │     • Morphological close (horizontal + vertical kernels)
  │     • Aspect ratio filter: h/w ≥ 2.0, height ≥ 200px, density ≥ 0.07
  │     • NMS (IoU 0.4)
  │
  └─► EDGE DAMAGE DETECTION
        • Extract left + right edge strips (12% of belt width)
        • Per-row 3-signal majority vote (≥ 2 of 3 must fire):
            S1: row residual > 1.6× strip median
            S2: row Sobel-X gradient > 1.6× strip median
            S3: row local std-dev > 1.6× strip median
        • Merge adjacent damaged rows (gap < 300px)
        • NMS (IoU 0.5)

---

Setup

pip install -r requirements.txt

GPU is optional. CPU works fine for pipeline.py and eval.py. GPU recommended only for classifier_train.py.

---

Running the Scripts

1. Main Inference — pipeline.py

Generates annotated images + detection JSONs for all input images.

python pipeline.py \
    --image_dir "images" \
    --output_dir "outputs_v3"

--label_dir is auto-detected as ../labels relative to image_dir. To specify explicitly:

python pipeline.py \
    --image_dir  "images" \
    --label_dir  "labels" \
    --output_dir "outputs_v3"

Output per image:

• outputs_v3/<name>.jpg — Annotated (🔴 scratch, 🟢 edge_damage)
• outputs_v3/<name>.json — Required detection format:

{
  "1": { "bbox_coordinates": [x_min, y_min, x_max, y_max] },
  "2": { "bbox_coordinates": [x_min, y_min, x_max, y_max] }
}

---

2. Dataset Analysis — train.ipynb

Open in Jupyter to analyse belt texture statistics and validate thresholds.

jupyter notebook train.ipynb

Outputs day/night texture statistics and recommended CFG threshold values.

---

3. Evaluation — eval.py

Computes Precision, Recall, F1, and estimated mF1@0.5–0.95 from the 136 manually reviewed images.

python eval.py \
    --review_xlsx "manual_review_SAMPLE.xlsx" \
    --output_dir  "outputs_v3" \
    --save_json   "metrics_report.json"

---

4. Auxiliary Classifier — classifier_train.py (optional)

Trains EfficientNet-B0 on the 136 manually reviewed images to predict damage type. Used only for auto-labelling the remaining 223 images — not part of the detection pipeline.

python classifier_train.py \
    --image_dir   "images" \
    --label_dir   "labels" \
    --review_xlsx "manual_review_SAMPLE.xlsx" \
    --output_dir  "classifier"

Val accuracy ~53.6% — limited by small training set (136 images for 4 classes).

---

Output Files Explained

File	Description
outputs_v3/	Primary deliverable — final annotated images + JSONs
detections_summary.xlsx	All detections from outputs_v3 compiled into Excel (3 sheets: all detections, per-image summary, stats dashboard)
manual_review_SAMPLE.xlsx	136 images manually reviewed as human ground truth
metadata.csv	Belt polygon coordinates extracted from all 359 label files
predictions_summary.csv	CNN-predicted damage types for remaining 223 images
metrics_report.json	Full evaluation report with per-class metrics

---

Performance Results

Image-Level Metrics (136 manually reviewed images)

Class	Precision	Recall	F1
Scratch	40.6%	80.8%	54.1%
Edge Damage	96.9%	70.5%	81.6%
Combined	62.2%	74.3%	67.7%

Estimated mF1@0.5–0.95

IoU Threshold	0.50	0.60	0.70	0.80	0.90	Mean
Scratch F1	54.1%	54.1%	50.9%	46.9%	43.3%	49.2%
Edge F1	81.6%	81.6%	77.3%	72.6%	67.7%	75.2%
Combined						62.2%

⚠️ mF1@0.5–0.95 is estimated via IoU simulation — no damage bbox ground truth available in the training set.

Detection Counts Across Versions

Version	Scratch	Edge	Change
V1 — Baseline	813	4	Initial run
V2 — CFG Tuned	603	33	Threshold tuning
V3 — Final	315	203	Aspect filter + 3-signal edge

---

Known Limitations

1. Scratch Precision 40.6% — clean belt texture triggers false positives on some image groups
2. No damage ground truth — mF1 is estimated, not ground-truth computed
3. Edge Recall 70.5% — subtle partial edge damage still occasionally missed
4. Day/Night auto-classification — based on brightness; borderline images may misclassify

---

Improvement Path

Approach	Expected mF1	Requirement
Current Classical CV (V3)	~62%	No additional data needed
YOLOv8 fine-tune on belt ROI	55–70%	Damage bbox labels required
Anomaly detection (PatchCore)	50–65%	Normal images + GPU
Ensemble CV + CNN	65–75%	Labels + GPU