合并文件中的重复项

#!/bin/bash

src=$1
dst=$2

uniq=`cat $src | awk '{print $1}' | sort | uniq`
for host in $uniq
do
    line=`cat $src | grep $host | awk -v host=$host 'BEGIN{count=0;}{if($1==host){count+=$2;}}END{if(count!=0) print host, count}'`
    echo $line >> $dst
done


 

eg：

abc 10

bcd 20

abc 10

after merge:

abc 20

bcd 20

注意上面得到uniq之前要进行sort，否则不能合并