Duplicate Combination Remover

An inhouse Python utility to remove duplicate combinations from an Excel (.xlsx) and CSV files (.csv) while keeping the first occurrence. It is a general purpose data cleaning utility in data preparocessing stage.

Features

• Supports Excel and CSV
• Remove duplicates using one or multiple columns combination
• Column selection by column number
• Saves:
  • *_unique.xlsx / csv
  • *_duplicates.xlsx / csv

Installation

git clone https://github.com/svmhub/Duplicate-Combination-Remover.git

cd Duplicate-Combination-Remover

pip install -r requirements.txt

Run

python duplicate_combination_remover.py

Example:

Enter file path:
data.xlsx

Enter column numbers:
3,13

Output:

• data_unique.xlsx
• data_duplicates.xlsx

Hence I have done this tool successfully! 🫰😍