Skip to content

DATASET_PROVENANCE.md

Latest commit

 

History

History
81 lines (59 loc) · 2.31 KB

DATASET_PROVENANCE.md

File metadata and controls

81 lines (59 loc) · 2.31 KB
Raw

Dataset Provenance

Public Dataset Reference

Primary dataset referenced by the notebook:

Kaggle Welding Defect - Object Detection
https://www.kaggle.com/datasets/sukmaadhiwijaya/welding-defect-object-detection

The dataset is not redistributed in this repository.

Expected Structure

The notebook expects a YOLO detection dataset with a data.yaml file and split folders similar to:

welding-defect-object-detection/
  data.yaml
  train/
    images/
    labels/
  valid/ or val/
    images/
    labels/
  test/
    images/
    labels/

YOLO label rows are expected to use normalized coordinates:

class_id x_center y_center width height

Notebook Dataset Resolution

The notebook resolves dataset paths in this order:

  1. Explicit WELD_DATASET_ROOT.
  2. Common local notebook folders.
  3. Kaggle and Colab-style mounted paths.
  4. KaggleHub cache.
  5. Optional KaggleHub download when enabled.

The notebook then writes a runtime-safe data_runtime.yaml into the artifact directory so downstream training uses explicit split paths.

Saved Audit Notes

The saved notebook run found:

  • 2,028 images.
  • 5,686 YOLO label rows.
  • 0 invalid labels.
  • 0 unreadable images.
  • 0 missing label files.
  • 4 exact cross-split duplicate image groups.
  • 1,529 border-touching boxes.
  • 286 tiny boxes.

These are workflow audit findings, not production acceptance results.

Data Governance Caveats

Before any production or commercial deployment:

  • Confirm the public dataset license and permitted usage.
  • Keep public-data experiments separate from plant-owned validation evidence.
  • Confirm that class names map to actionable manufacturing dispositions.
  • Freeze a plant-specific annotation guide.
  • Require quality-engineering signoff for label taxonomy and edge cases.
  • Validate on plant-specific images from the target station, lighting, fixture, camera, supplier, and weld process.

Recommended Production Dataset Controls

  • Use a locked plant holdout set.
  • Remove duplicate leakage across splits.
  • Track image source, station, camera, shift, part family, weld program, and label reviewer.
  • Version labels and annotation guidelines.
  • Preserve image-quality diagnostics for drift comparison.
  • Maintain an active-learning queue for low-confidence cases, operator overrides, new defect modes, and confirmed false accepts.