初识sed和gwak

一、sed编辑器
sed命令的格式如下：
sed options script file
选项
-e script        在处理输入时，将script中指定的命令添加到已有的命令中
-f file            在处理输入时，将file中指定的命令添加到已有的命令中
-n                不产生命令输出，使用print命令来完成输出

1.在命令行定义编辑器命令

$ echo "This is a test" | sed '/test/big test/'
This is a big test

[root@localhost sed]# vim data1.txt
[root@localhost sed]# sed 's/dog/cat/' data1.txt
The quick brown fox jumps over the lazy cat.
The quick brown fox jumps over the lazy cat.
The quick brown fox jumps over the lazy cat.
The quick brown fox jumps over the lazy cat.
The quick brown fox jumps over the lazy cat.
The quick brown fox jumps over the lazy cat.
The quick brown fox jumps over the lazy cat.
[root@localhost sed]# cat data1.txt
The quick brown fox jumps over the lazy dog.
The quick brown fox jumps over the lazy dog.
The quick brown fox jumps over the lazy dog.
The quick brown fox jumps over the lazy dog.
The quick brown fox jumps over the lazy dog.
The quick brown fox jumps over the lazy dog.
The quick brown fox jumps over the lazy dog.

2.在命令行使用多个编辑器命令

[root@localhost sed]# sed -e 's/brown/green/; s/dog/cat/' data1.txt
The quick green fox jumps over the lazy cat.
The quick green fox jumps over the lazy cat.
The quick green fox jumps over the lazy cat.
The quick green fox jumps over the lazy cat.
The quick green fox jumps over the lazy cat.
The quick green fox jumps over the lazy cat.
The quick green fox jumps over the lazy cat.
[root@localhost sed]# cat data1.txt
The quick brown fox jumps over the lazy dog.
The quick brown fox jumps over the lazy dog.
The quick brown fox jumps over the lazy dog.
The quick brown fox jumps over the lazy dog.
The quick brown fox jumps over the lazy dog.
The quick brown fox jumps over the lazy dog.
The quick brown fox jumps over the lazy dog.

3.从文件中读取编辑器命令

[root@localhost sed]# cat script1.sed
s/brown/green/
s/fox/elephant/
s/dog/cat/
[root@localhost sed]# sed -f script1.sed data1.txt
The quick green elephant jumps over the lazy cat.
The quick green elephant jumps over the lazy cat.
The quick green elephant jumps over the lazy cat.
The quick green elephant jumps over the lazy cat.
The quick green elephant jumps over the lazy cat.
The quick green elephant jumps over the lazy cat.
The quick green elephant jumps over the lazy cat.

二、gawk程序可以
①定义变量来保存数据；
②使用算数和字符串操作符处理数据；
③使用结构化编程概念(if-then)来为数据处理增减处理逻辑；
④通过提取数据文件中的数据元素，将其重新排列或格式化，生成格式化报告。

gawk是awk语言中使用gnu的一种

2.1 gawk命令格式
gawk options program file
选项
-F fs                                   指定行中划分数据字段分隔符
-f file                                 从指定的文件中读取程序
-v var=value                      定义gawk程序中的一个变量及其默认值
-mf N                                指定要处理的数据文件中的最大字段数
-mr N                                指定数据文件中的最大数据行数
-W keyword                       指定gawk的兼容模式或警告等级

2.2 从命令行读取程序脚本

[root@localhost sed]# gawk '{print "Hello World!"}'
This is a test
Hello World!
hello
Hello World!
This is another test
Hello World!

Hello World!
^C

2.3 使用数据字段变量
$0代表整个文本行
$1代表文本行中的第1个数据字段
$2代表文本行中的第2个数据字段
$n代表文本行中的第n个数据字段

[root@localhost gawk]# cat data2.txt
One line of test text.
Two line of test text.
Three line of test text.
[root@localhost gawk]# gawk '{print $1}' data2.txt
One
Two
Three

[root@localhost gawk]# gawk -F: '{print $1}' /etc/passwd
root
bin
daemon
adm
lp
sync
shutdown
halt
mail
uucp
operator
games
gopher
ftp
nobody

如果不是空格分隔符，则用-F指定

2.4 在程序中使用多个命令

[root@localhost gawk]# echo "My name is Rich" | gawk '{$4="Christine"; print $0}'
My name is Christine

2.5 从文件中读取程序

[root@localhost gawk]# cat script2.gawk
{print $1 "'s home directory is " $6}
[root@localhost gawk]# gawk -F: -f script2.gawk /etc/passwd
root's home directory is /root
bin's home directory is /bin
daemon's home directory is /sbin
adm's home directory is /var/adm
lp's home directory is /var/spool/lpd
sync's home directory is /sbin
shutdown's home directory is /sbin
halt's home directory is /sbin
mail's home directory is /var/spool/mail
uucp's home directory is /var/spool/uucp
operator's home directory is /root
games's home directory is /usr/games
gopher's home directory is /var/gopher

[root@localhost gawk]# cat script3.gawk
{
text = "'s home directory is "
print $1 text $6
}
[root@localhost gawk]# gawk -F: -f script3.gawk /etc/passwd
root's home directory is /root
bin's home directory is /bin
daemon's home directory is /sbin
adm's home directory is /var/adm
lp's home directory is /var/spool/lpd
sync's home directory is /sbin
shutdown's home directory is /sbin

2.6 在处理数据前运行程序脚本

[root@localhost gawk]# gawk 'BEGIN {print "Hello World!"}'
Hello World!

[root@localhost gawk]# cat data3.txt
Line 1
Line 2
Line 3
[root@localhost gawk]# gawk 'BEGIN {print "The data3 File Contents:"}
{print $0}' data3.txt
The data3 File Contents:
Line 1
Line 2
Line 3

2.7 在处理数据后运行脚本

[root@localhost gawk]# gawk 'BEGIN {print "The data3 File Contents:"}
> {print $0}
> END {print "End of file"}' data3.txt
The data3 File Contents:
Line 1
Line 2
Line 3
End of file

[root@localhost gawk]# cat script4.gawk
BEGIN {
print "The latest list of users and shells"
print " UserID \t shell"
print "-------- \t -------"
FS=":"
}

{
print $1 "      \t  "    $7
}

END {
print "This concludes the listing"
}
[root@localhost gawk]# gawk -f script4.gawk /etc/passwd
The latest list of users and shells
UserID 	 shell
-------- 	 -------
root      	  /bin/bash
bin      	  /sbin/nologin
daemon      	  /sbin/nologin
adm      	  /sbin/nologin
lp      	  /sbin/nologin
sync      	  /bin/sync
。。。
nfsnobody         /sbin/nologin
This concludes the listing