How to use the CSV Duplicate Remover
- 1. Load your CSVChoose a file or paste the text; commas, semicolons, tabs and pipes are detected.
- 2. Choose the rulesPick the columns that define a duplicate, and whether to ignore spaces and case.
- 3. Remove duplicatesDownload the cleaned file and see exactly which rows were dropped.
How duplicates are found
Two rows count as duplicates when their values match in the columns you tick, or in every column if you tick none. With Ignore extra spaces on, " asha verma " matches "Asha Verma" once case is ignored too. Those rules apply only when comparing; the cleaned file keeps each value exactly as it was. The first copy is kept by default (or the last, if you choose), and the original row order is preserved. Files are read by your browser and never uploaded.
Choosing columns matters. Compare on Email only and two enquiries from the same person count as duplicates even if their phone numbers differ. Compare whole rows and only exact repeats are removed.
Worked example
The sample has 10 enquiry rows. Comparing whole rows with spaces and case ignored, the tool finds 3 duplicates and keeps 7 unique rows. The removed rows are " asha verma " (which matches Asha Verma once spaces and case are ignored), a second Ravi Kumar and a second Neha Singh. The cleaned file downloads straight away, and the removed rows can be downloaded separately to check.
Good to know
- Our duplicate logic matches pandas'
drop_duplicateson 30 test files, including keep-first and keep-last. - Quoted fields with commas or line breaks inside are handled, following the CSV standard (RFC 4180).
- Files up to about 50 MB work; very large files depend on your device's memory.
Frequently asked questions
Will it change my data?
No. Apart from dropping duplicate rows, every value is written back exactly as it was.
Can it handle Excel files?
Save the sheet as CSV first, or use our Excel to CSV converter.
Is the CSV duplicate remover free?
Yes. You can open the CSV duplicate remover, load files and set every option without signing up. When you remove duplicates for the first time, we ask once for your name and WhatsApp number; after that it, and every other Uncodemy tool, opens straight away.