Contactless Device OCR Pipeline (Proof of Concept)

Note

Education and Portfolio Only This repository is developed as a Proof of Concept (POC) for a client. It is not intended for production environments and is maintained strictly for educational and portfolio purposes.

This project implements a two-stage deep learning pipeline for contactless device reading:

1. Text Detection (textbox/): A PyTorch implementation of the TextBoxes architecture (based on SSD with a VGG-16 backbone) that detects the bounding boxes of digits/text in an image.
2. Text Recognition (ocr/): A TensorFlow/Keras implementation of a CRNN (Convolutional Recurrent Neural Network) utilizing CTC (Connectionist Temporal Classification) loss to decode sequences of characters (digits) from cropped bounding box images.

---

Pipeline Architecture

                       +-------------------+
                       |    Input Image    |
                       +---------+---------+
                                 |
                                 v
                       +---------+---------+
                       |  TextBoxes Model  |  (textbox/detect_device.py)
                       |  (PyTorch/VGG16)  |
                       +---------+---------+
                                 |
                                 v
                       +---------+---------+
                       | Cropped Digit Box |
                       +---------+---------+
                                 |
                                 v
                       +---------+---------+
                       |    CRNN Model     |  (ocr/detect.py)
                       | (TensorFlow/CTC)  |
                       +---------+---------+
                                 |
                                 v
                       +---------+---------+
                       |  Recognized Text  |
                       +-------------------+

---

Folder Structure

.
├── textbox/                  # Stage 1: Text Bounding Box Detection (PyTorch)
│   ├── textboxes.py          # TextBoxes model construction
│   ├── detect_device.py      # Run detection on test images & save crops
│   ├── train.py              # Model training script
│   ├── data/                 # Datasets loaders (ICDAR, COCO, SynthText)
│   ├── layers/               # Custom SSD/TextBoxes loss & utility layers
│   ├── utils/                # Augmentation & helper routines
│   └── Dataset/              # Test and training image directories
│
└── ocr/                      # Stage 2: Character/Digit recognition (TensorFlow/Keras)
    ├── model.py              # CRNN model definition
    ├── train_ocr.py          # OCR training loop
    ├── detect.py             # Inference on cropped text images
    ├── utils.py              # CTC decoding, label mapping & preprocessing
    └── weights/              # OCR trained checkpoints (weights_old.h5)

---

Installation & Setup

Local Setup

Ensure you have Python 3.8+ installed. It is highly recommended to run each module in its own virtual environment since they use different ML frameworks (PyTorch vs. TensorFlow).

1. TextBoxes Setup (Detection)

cd textbox
python -m venv venv
# Windows:
venv\Scripts\activate
# Unix/macOS:
source venv/bin/activate

pip install -r requirements.txt

2. OCR Setup (Recognition)

cd ocr
python -m venv venv
# Windows:
venv\Scripts\activate
# Unix/macOS:
source venv/bin/activate

pip install -r requirements.txt

---

Usage

1. Running Text Detection

Activate the textbox virtual environment and run the test script:

cd textbox
python detect_device.py

This script reads images from ./Dataset/train_images_new/, performs bounding box localization using the checkpoint inside ./runs/, crops the detected text bounding boxes, and writes them to the directory as crop_X.jpg.

2. Running Digit Recognition

Activate the ocr virtual environment and run:

cd ocr
python detect.py

This reads ./crop_1.jpg, preprocesses the image, runs the CRNN recognition model using ./weights/weights_old.h5, decodes the outputs with greedy CTC decoding, and prints the predicted digit string to standard output.

---

Verification & Status

• Correctness: Verified and fixed major bugs including typo handlers (except Exception), conditional PyTorch Generator fallback for non-CUDA hardware, and path corrections for OCR model weight checkpoints (weights_old.h5).
• Docker Support: Skipped. The project operates as isolated scripts for model training/evaluations and does not package multi-container microservices or deployment APIs.
• Environment Configuration: None required. System operates entirely on local filesystem folders and datasets.