背景：一文档多个文件，查重，有多个字段，类似下面
11 john she
22 mybe he
33 lili he
11 john he
33 sanjiu he
根据第一列字段 去重，筛选出文件
cat 0000026561011720230101.txt | awk -F "," '{print $1}' | sort -n | uniq |wc -l
cat 0000033778011820230311.txt| awk -F "," '{print $1}' | sort -n | uniq |wc -l
for i in `ls /record/cdr/00000337780118`; do cat $i |awk -F "," '{print $1}' | sort -n | uniq |wc -l >> /root/result2.log; done
sort -k 1,1 -u 1.txt
输出结果为
11 john she
22 mybe he
33 lili he