那阵子在moto做项目，利用间隙看完了一本o'reilly公司的sed&awk介绍，作了一些笔记。巧的是所学的一些东西在项目后期居然用到非常多，呵呵
1. 关于过滤字符串问题
比如在一个文本中搜索abcd这四个字符，使用grep 结果出来abcdef 、zabcd 、abcd 、 hiabcd.... 等等，其实只想搜索abcd
A: 使用精确匹配 $grep "" filename
2. sed中查找模式匹配：
. 单字符通配符
[0-9] 匹配0,1,2,3...8,9
[a-z] 匹配a,b,c...z
[^0-9] 匹配不含0,1,2...9的所有其他字符
".*" 匹配""内任何字符串
^ 行开始标志
$ 行结束标志
3. sed中用bbb替换同一行中包括字符串and的字符串aaa，而不是每一行中的字符串aaa
A: $sed -e '/and/s/aaa/bbb/' filename
4. sed删除文件filename中的所有行
A: $sed d filename
5. sed删除文件filename中的2至5行
A: $sed 2,5d filename
6. sed删除文件filename中包含字符串abc的所有行
A: $sed /abc/d filename
7. A regular expression can use "n" to match an embeded newline
8. The regular expression can be delimited by any character except a blank or a newline
EX: s!/usr/mail!/usr2/mail!
Note that the delimiter appears three times and is required after the replacement "/usr2/mail"
9. SED Insert: [line-address]i
sed '//i
4700 cross court
French Lick, IN' filename
Desc: inserts two lines of text at a line matching ""
10. SED Append: [line-address]a
sed '$a
Append a new line ' filename
DESC: append a new line in the end of filename
11. SED Change: [line-address]c
sed '/100/c
one hundred' filename
DESC: change the whole lines containing '100' into 'one hundred'
12. 用SED显示TAB等特殊字符: l
sed -n -e "l" test1
NOTE:显示test1文件的内容，包括TAB等特殊字符。-n过滤重复行
13. SED n:下一行/next p:显示/print
$ sed -n -e '/James/{
> n
> p
> }' filename
DESC: 显示filename中含有James字符串的下一行
14. SED q: quit
$sed '100q' filename
DESC: print the first 100 lines from filename
15. SED y: transform
/.*/y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/
DESC:replace all the characters into capital
16. convert file 'filename' from 1 to 2
2 1
11 22
22 11
111 222
222 111
$ sed '/1/{
> h
> d
> }
> /2/{
> G
> }' filename
h/H: Hold, copy or append contents of pattern space to hold space
g/G: Get, copy or append contents of hold space to pattern space
x: Exchange, swap contents of hold space and pattern space
17. AWK中的特殊字符
a alert character, usually ASCII BEL character
b backspace
f formfeed
n newline
r carriage return
t horizontal tab
v vertical tab
c any literal character c
18. AWD系统变量
FS field separator
NR the number of current input record
FILENAME the name of current input file
RS the record separator, as a newline
NF the number of fields for the current record
EXAMPLE awkcmd 列出各纪录平均值，并标出行号，最后显示文件名和记录数:
#list the avg scores and row numbers
BEGIN { FS=" " }
{ print NR,".",$1,",",($2+$3+$4)/3 }
END { print
print "File " FILENAME " has " NR " rows " }
$awk -f awkcmd sourcefile
19. #将多行的块纪录转成单行纪录
BEGIN { FS="n"; RS="" }
{ print $1, $NF }
20. 显示当前目录下各文件名和大小的例子
ls -lR|awk '
BEGIN {print " size"," ", "filename"}
NF==9 && /^-/ { #9列并以-开头表示文件
filesize += $5
++filenum
print " file ", $5," ",$9
}
NF==9 && /^d/ { #9列并以d开头表示子目录
print "
", "t", $9
}
$1 ~ /^..*:$/ { # ~: Match Operator
#ls -lR中子目录显示为 ./subdir: 以"."开头"$"结尾的行判断为子目录的行
print "t" $0
}
END { print "Total: ", filesize, " bytes (", filenum, " files)"} '
21. AWK传递参数和检索数据
$ cat awk_par
awk '$2==search' search=$1 srcfile
运行:
$ awk_par par_val
DESC: 参数par_val传递给变量search，显示srcfile中第二列为par_val的记录行
22. AWK中将shell参数传递给awk
search=$1
awk '$2 ~ /'"${search}"'/' srcfile
DESC: 将shell的第一个参数传递给变量search，同srcfile中第二列比较
如果改成 awk '$2 ~ /'"${search:-.*}"'/' srcfile，则如果shell运行时无参数$1,就显示所有记录
