YOLO Label Recovery

Engineering portfolio: Architecture | Reproducible demo | Collaboration platform | Evidence boundaries

English | 简体中文 | Chinese documentation hub

A safe, auditable and memory-efficient multi-teacher annotation recovery and human-review platform for YOLO datasets.

This project was extracted from an industrial safety-vision workflow. It uses one detector per class to scan a multi-class dataset, identifies high-confidence predictions that are not covered by existing labels, and writes additions to a new label tree without modifying the source dataset.

The product view above comes from the running Vue + Spring Boot application and one user-approved underground review image. It groups five candidates in context while exposing the lease, constrained decisions, shortcuts and release path.

Interview quick tour

Time	Start here	What it demonstrates
30 seconds	Core properties and the screenshots below	Business problem, system boundary and product form
5 minutes	Architecture	End-to-end candidate lifecycle and safety constraints
15 minutes	Reproducible demo	Model-free audit, review packaging and safe apply
30 minutes	Code walkthrough	Python streaming pipeline, Spring transactions, MySQL constraints and Vue state
Deep dive	Mining-system design	Offline data governance connected to RTSP monitoring, alerting, RAG and agents

Pre-generated showcase

No GPU or live command is needed to inspect these results. Analytical report screenshots use committed synthetic fixtures; the grouped-review gallery and web-platform captures use real, user-approved production review samples to show the actual operating experience. Screenshots demonstrate behavior and report structure, not model accuracy claims.

Model-free dataset audit

The fixture intentionally contains one invalid class ID, one orphan label and one exact image duplicated across train/val. The audit correctly returns FAIL, 2 critical issues, 1 warning and 1 cross-split duplicate group.

Multi-teacher recovery report

Evidence	Pre-generated result
Image-model scans	3,600
Specialist teachers	3
AUTO / REVIEW examples	3 / 3
Initial batch	32
Stable batches	32 / 16 / 8
Simulated OOM retries	3
Source labels modified	No

Audited threshold calibration

The public fixture contains 2,400 reviewed candidates across all six classes. AUTO requires the 95% Wilson lower confidence bound for precision to reach 95%, while REVIEW preserves 90% of audited positives. All six classes produce statistically supported policies. AUTO thresholds range from 0.732 for tractor to 0.859 for smoking, demonstrating why one global confidence threshold is unsafe.

Cross-Teacher consensus gate

The public fixture contains 96 primary candidates across six classes. Of 72 primary AUTO candidates, 48 receive one-to-one spatial support from an independent verifier and remain AUTO; 24 are safely downgraded to REVIEW. The stage is model-free and adds no GPU memory pressure.

Perceptual near-duplicate review groups

The public fixture groups resize, JPEG recompression and brightness variants without merging black and white low-texture frames. It finds 3 groups containing 7 images, reduces first-pass review to 3 representatives and flags 2 groups crossing dataset splits.

Diversity-aware active review queue

The public fixture contains 36 imbalanced REVIEW images. A budget of 12 covers all 6 classes, with one class represented in each of the first six positions. Dynamic rarity prevents small classes from being ignored while perceptual diversity suppresses repeated frames.

Exhaustive GT/AUTO human-review gate

The no-GPU fixture enumerates all four image/class states (GT0_AUTO0, GT1_AUTO0, GT0_AUTO1, GT1_AUTO1). It combines IoU, intersection-over-smaller-area, normalized center distance and area ratio to distinguish already-labelled objects, same-target extent disagreement, distinct missing objects and cross-class conflicts. High confidence remains evidence, not permission to write a label.

Image-grouped desktop review application

The portable Tk application groups every candidate belonging to the same source image. Reviewers can compare overlapping classes in context, switch between boxes without reloading the image, and only choose actions permitted by the decision engine. It supports Chinese and English interfaces, keyboard review, append-only JSONL journaling, periodic atomic CSV checkpoints and crash-safe resume. The gallery above comes from a real production review package and includes multi-class, multi-box joint scenes.

See Grouped Review Application for the queue contract, persistence model and deployment workflow.

Multi-user collaboration platform

The repository now includes a deployable web platform for teams that outgrow the portable desktop reviewer. A Vue 3 + TypeScript client talks to a Spring Boot 4 REST API backed by MySQL and Flyway. JWT authentication, ADMIN / REVIEWER / AUDITOR RBAC, project membership, image-level pessimistic claiming, renewable leases, optimistic versions and immutable audit events prevent duplicate work and stale decisions. Claiming one candidate leases every pending candidate from the same (project, split, image) to one reviewer, preserving full-image context and preventing two people from making inconsistent decisions on different boxes from one image. One-click decisions automatically advance through the owned image group and then to the next image; visible lease/network health and a reviewer-scoped recent-decision correction path make failures recoverable without weakening the audit boundary. The Python bridge streams the existing review_queue.csv into the API in bounded, idempotent batches.

These are captures of the actual Vue + Spring Boot application, not design mockups. The primary workspace shown at the top deliberately uses one real underground image with five grouped candidates (3 person + 1 vest + 1 tractor), so the candidate rail, full image canvas, lease state, confidence, constrained decisions, keyboard shortcuts and release path are visible in one frame. The login view shows the role-aware entry used by reviewers, while the admin view demonstrates account creation and project membership assignment.

The earlier Tk desktop application and the web platform are complementary rather than redundant: the desktop version remains a portable offline handoff tool, while the web version adds identity, atomic image-level claiming, renewable group leases, project isolation, version conflicts and immutable audit history for team review.

flowchart LR
    P["Python Multi-Teacher pipeline"] -->|"review_queue.csv / batches <= 500"| A["Spring Boot API"]
    V["Vue review workspace"] -->|"JWT + heartbeat + decisions"| A
    A --> M[("MySQL + Flyway")]
    A --> R["Read-only review visuals"]

Loading

The workflow was validated on a trusted campus LAN with two independent reviewer accounts and then completed end to end. All 30,183 / 30,183 candidates reached a final decision with 0 pending, claimed or escalated tasks: 15,283 ACCEPT_ADD, 6,307 ACCEPT_EVAL_LABEL, 551 ACCEPT_REPLACE_GT and 8,042 REJECT. The closeout pipeline stopped writes, exported decisions by stable candidate_id, created a transactionally consistent MySQL dump, rejected incomplete or uncertain states, and generated SHA-256 checksums plus an exact-coordinate class-remap manifest. These figures demonstrate workflow completion and traceability, not detector accuracy or maximum concurrency.

See Collaboration Platform for screenshots, deployment, API contracts, concurrency design and interview walkthrough.

See Review Completion Case Study for the image-lock design, final decision distribution, immutable export gate and 5090 retraining handoff.

Production-scale validation

The review gate was also exercised on a private six-class dataset containing 29,071 images. Six specialist Teachers completed 174,426 image-model passes, produced 99,696 evidence rows and generated 30,183 review rows with 0 render failures while leaving the source labels unchanged. See the anonymized case study.

Why this project exists

Multi-class datasets often contain combined scenes such as person + helmet + smoking or person + slipper. If the original annotation process focused on one target at a time, valid objects from other classes can be missing. Training a new multi-class model on incomplete labels can make the model learn the wrong supervision signal.

The workflow is deliberately conservative:

flowchart TD
    A["Existing YOLO dataset"] --> Q["Model-free audit"]
    Q -->|"schema and split checks pass"| B["Immutable source labels"]
    B --> C["Load one specialist teacher"]
    C --> D["Batch FP16 streaming inference"]
    D --> E["Same-class prediction/GT IoU matching"]
    E -->|"IoU >= existing threshold"| F["Already labeled: ignore"]
    E -->|"Possible missing object"| G["Confidence routing"]
    G --> H["IGNORE"]
    G --> I["REVIEW + audit CSV"]
    G --> J["AUTO candidate"]
    J --> M{"Optional verifier agreement"}
    M -->|"supported evidence"| R["Exhaustive GT/AUTO review gate"]
    M -->|"unsupported"| I
    I --> R
    R -->|"explicit human decision"| K["Immutable derived label tree"]
    K --> N["Trainable YOLO dataset"]
    I --> L["HTML report and class-wise samples"]
    J --> L
    R --> L

Loading

Main properties

• Original labels/ are read-only from the tool's point of view.
• Only one detector is kept in GPU memory at a time.
• stream=True consumes prediction results incrementally.
• Candidate records are written incrementally to CSV instead of accumulating all predictions in RAM.
• Per-class model class IDs are mapped to the dataset class IDs from data.yaml.
• Existing-label IoU and candidate-duplicate IoU are separate controls.
• --materialize-dataset creates a standard YOLO dataset using hardlinks when possible.
• Review images are grouped by unique image so multiple candidates from one image remain visible together.
• A model-free audit catches malformed labels, corrupt images and exact train/val/test leakage before GPU work starts.
• Audited candidate decisions can calibrate class-specific AUTO policies using a Wilson precision lower bound and REVIEW policies using positive recall.
• Independent Teacher candidate streams can gate AUTO decisions with one-to-one spatial agreement without loading two models together.
• Perceptual hashes, a BK-tree and conservative visual guards group repeated review work and expose near-duplicate split leakage.
• Image-level active review combines confidence entropy, dynamically decayed class rarity and greedy perceptual diversity.
• Exhaustive GT/AUTO accounting prevents candidate-only reports from hiding absence cases.
• Offline review requires explicit add, replace, evaluation or reject decisions and autosaves progress.
• The desktop reviewer groups candidates by image, supports Chinese/English UI and disables actions that are invalid for the current decision case.
• Every click is first persisted to an append-only JSONL journal; CSV snapshots use atomic replacement for safe recovery after interruption.
• Safe apply blocks unresolved decisions, detects source-GT drift, rechecks duplicates and creates an immutable derived dataset.
• Every scan records a local manifest with parameters, image inventory, package versions, CUDA and GPU metadata.

One-minute public demo

The demo deliberately creates an invalid class ID, an orphan label and an image duplicated across splits. No model weights or private data are needed.

python examples\create_synthetic_dataset.py --output .demo-dataset
yolo-label-recovery audit .demo-dataset `
  --output-dir .demo-audit `
  --hash-images `
  --check-images

Open .demo-audit\dataset_audit.html. The expected result is FAIL: the fixture proves that the audit catches the injected defects.

Inspect an unfamiliar training machine before an expensive scan:

yolo-label-recovery doctor --output environment.json --redact-paths

Quick start

Install only the lightweight audit/report tools (no PyTorch or Ultralytics download):

python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -e .

yolo-label-recovery audit D:\data\mining-safety --output-dir D:\data\audit-001 --hash-images --check-images

Calibrate thresholds from a human-reviewed candidate CSV without any GPU dependency:

yolo-label-recovery calibrate reviewed_candidates.csv `
  --output-dir calibration `
  --target-auto-precision 0.95 `
  --auto-confidence-level 0.95 `
  --target-review-recall 0.90 `
  --min-auto-samples 20 `
  --redact-paths

Gate primary AUTO candidates with an independent verifier, without a GPU dependency:

yolo-label-recovery consensus primary_candidates.csv verifier_candidates.csv `
  --output-dir consensus `
  --agreement-iou 0.50 `
  --verifier-min-confidence 0.50 `
  --redact-paths

Group perceptual near-duplicates without loading a model or changing source data:

yolo-label-recovery cluster D:\data\mining-safety `
  --output-dir D:\data\near-duplicate-audit `
  --workers 4 `
  --max-distance 6 `
  --redact-paths

Build a limited-budget, diversity-aware human review queue:

yolo-label-recovery prioritize D:\runs\candidates_review.csv D:\data\mining-safety `
  --output-dir D:\runs\priority-review `
  --budget 500 `
  --redact-paths

Build an exhaustive offline review bundle from Teacher candidate evidence:

python examples\create_review_fixture.py --output-dir .demo-review-fixture
yolo-label-recovery review-build .demo-review-fixture\dataset .demo-review-fixture\candidates.csv `
  --output-dir .demo-review-result `
  --render `
  --redact-paths

After every row has an explicit human decision, create a separate reviewed dataset:

yolo-label-recovery review-apply D:\data\mining-safety D:\runs\company-review\company_decisions.csv `
  --output-root D:\data\mining-safety-reviewed

For GPU-assisted label recovery, first install the CUDA-compatible PyTorch build required by the target GPU, then install the inference extra:

python -m pip install -e ".[inference]"

yolo-label-recovery run `
  --dataset-root D:\data\mining-safety `
  --out-root D:\data\autolabel_run_001 `
  --models-json configs\models.example.json `
  --classes person helmet vest tractor slipper smoking `
  --splits train val test `
  --imgsz 832 `
  --batch 32 `
  --device 0 `
  --workers 0 `
  --draw-review `
  --draw-auto-samples 80 `
  --adaptive-batch `
  --dry-run `
  --force

Use --dry-run first. It generates statistics, CSV candidates and review images but does not write automatic additions. After checking the output, remove --dry-run and add --materialize-dataset if a trainable dataset is required.

Generate the visual quality report after a run:

yolo-label-recovery report D:\data\autolabel_run_001

Add --redact-paths when generating a report for GitHub or an interview portfolio. Real run manifests intentionally retain local dataset/model paths for reproducibility and should not be published without review.

Resume an interrupted long-running scan with the original arguments, replacing --force with --resume:

yolo-label-recovery run <same arguments> --resume

The checkpoint stores the committed image cursor and statistics after every successful batch. Candidate CSV and label writes are idempotent, so an interrupted batch can be retried without duplicating rows or labels.

Expected dataset format

dataset-root/
  data.yaml
  images/
    train/
    val/
    test/
  labels/
    train/
    val/
    test/

data.yaml must define names in the same order as the label class IDs. The tool validates the class names before scanning.

Output format

out-root/
  labels_autofill_v1/       # original labels plus AUTO additions
  candidates_auto.csv       # high-confidence candidates
  candidates_review.csv     # medium-confidence candidates
  candidates_all.csv        # complete candidate audit stream
  auto_samples/<class>/     # sampled AUTO images
  review_images/<class>/    # sampled REVIEW images
  summary.json
  summary.txt
  state.json                # atomic resume checkpoint
  manifest.json             # arguments, inventory, packages, CUDA and GPU
  report.html               # generated quality report
  trainable_dataset/        # optional, created by --materialize-dataset
    data.yaml
    images/
    labels/

The source label tree is never used as an output path. Delete the output directory to discard an experiment and rerun from the untouched source dataset.

Default thresholds

Class	AUTO	REVIEW
person	0.75	0.55
helmet	0.75	0.55
vest	0.75	0.55
tractor	0.70	0.50
slipper	0.65	0.45
smoking	0.65	0.40

Override a class with --threshold smoking:0.70:0.45. The format is class:auto_threshold:review_threshold.

Resource model

For N images and K single-class models, the compute work is approximately K x N image-model inferences. The implementation does not load all images or all models at once:

• GPU: current model, current batch activations, current prediction tensors.
• CPU RAM: image paths, current batch decode objects, current-class label cache, bounded review samples.
• Disk: streamed CSV rows and the output label copy.

With --adaptive-batch, the tool reports the failing class, split and batch size, discards the uncommitted current batch, and retries it at half the batch size. The current batch is committed to CSV and labels only after successful candidate generation. The summary and HTML report show initial/stable batch sizes and OOM retry counts.

Project status

This repository is a cleaned engineering artifact, not a released benchmark. Real project images, annotation files, model weights, logs and machine-specific paths are intentionally excluded. Reproducible evaluation requires a user-provided YOLO dataset and single-class weights.

See:

• Chinese documentation hub
• Reproducible public demo
• Core code walkthrough
• Evidence and claim boundaries
• Review completion case study
• Post-review safe-apply runbook
• Mining-safety system design
• Interview Q&A
• Architecture and workflow
• Memory and GPU design
• Data governance
• Threshold calibration
• Threshold calibration (Simplified Chinese)
• Cross-Teacher consensus
• Cross-Teacher consensus (Simplified Chinese)
• Perceptual near-duplicate grouping
• Perceptual near-duplicate grouping (Simplified Chinese)
• Active review prioritization
• Active review prioritization (Simplified Chinese)
• Exhaustive GT/AUTO human review
• Exhaustive GT/AUTO human review (Simplified Chinese)
• Production-scale validation
• Production-scale validation (Simplified Chinese)
• Interview presentation
• Portfolio and interview guide
• Portfolio and interview guide (Simplified Chinese)
• Detection model card template
• Architecture decisions
• Roadmap
• Contributing

Verification

The repository includes dependency-light smoke tests that do not require pytest:

python -m py_compile autolabel_with_single_class_models.py
python tests\run_smoke_tests.py
pytest

For a complete development environment, use python -m pip install -e ".[inference,dev]".