Quick Python way of finding duplicate records in a text file and excel coulmn
Purpose / Task : Automated task of finding duplicate records in a text file or excel coulmn using simple Python pandas script.
USAGE: Before running the file ./find_duplicate.py read the comments and change the appropriate values. Duplicate values will be logged in the log file. Sample inputs are also attached for demonstration.
Dependencies : Requires python pandas,codecs,sys,os libraries.