DeMix is an automated framework designed for training data debugging with mixed error types. First, DeMix computes influence vectors for training samples. The influence vectors are then fed into a trained Data Error Classifier (DEC), which identifies suspicious samples and diagnoses their specific error types. Subsequently, DeMix executes type-specific repair on the detected samples.
- Install dependencies:
# create conda environment
conda create -n demix python=3.9 -y
conda activate demix
# install dependencies
pip install torch --index-url https://download.pytorch.org/whl/cu118
pip install scikit-learn numpy pandas ucimlrepo tqdm matplotlib-
Create a new directory
demix_filesto store the DEC checkpoints and the dataset. -
Download DEC checkpoints from google drive and place them in the
demix_files/dec_ckpts/directory. -
The dataset used for DeMix will be automatically downloaded and saved in the
demix_files/data/directory.
The script automatically inject errors (with a clean ratio of alpha) to the dataset, perform data debugging and repair with DeMix, and evaluate the trained model performance after repair.
python scripts/repair_data.py --save_dir demix_files --data_name adult --model_name mlp1 --alpha 0.5