sed、awk

 

1. 关于过滤字符串问题
比如在一个文本中搜索abcd这四个字符，使用grep 结果出来abcdef 、zabcd 、abcd 、 hiabcd.... 等等，其实只想搜索abcd
A: 使用精确匹配 $grep "" filename

 

2. sed中查找模式匹配：
. 单字符通配符
[0-9] 匹配0,1,2,3...8,9
[a-z] 匹配a,b,c...z
[^0-9] 匹配不含0,1,2...9的所有其他字符
".*"   匹配""内任何字符串
^  行开始标志
$  行结束标志

 

3. sed中用bbb替换同一行中包括字符串and的字符串aaa，而不是每一行中的字符串aaa
A: $sed -e '/and/s/aaa/bbb/' filename

 

4. sed删除文件filename中的所有行
A: $sed d filename

 

5. sed删除文件filename中的2至5行
A: $sed 2,5d filename

 

6. sed删除文件filename中包含字符串abc的所有行
A: $sed /abc/d filename

 

7. A regular expression can use "n" to match an embeded newline

 

8. The regular expression can be delimited by any character except a blank or a newline
   EX: s!/usr/mail!/usr2/mail!
   Note that the delimiter appears three times and is required after the replacement "/usr2/mail"

 

9. SED Insert: [line-address]i
   sed '//i
   4700 cross court
   French Lick, IN' filename
   Desc: inserts two lines of text at a line matching ""
  
10. SED Append: [line-address]a
    sed '$a
    Append a new line ' filename
    DESC: append a new line in the end of filename  
   
11. SED Change: [line-address]c
    sed '/100/c
    one hundred' filename
    DESC: change the whole lines containing '100' into 'one hundred'  

 

12. 用SED显示TAB等特殊字符: l
    sed -n -e "l" test1
    NOTE:显示test1文件的内容，包括TAB等特殊字符。-n过滤重复行

 

13. SED n:下一行/next p:显示/print
    $ sed -n -e '/James/{
    > n
    > p
    > }' filename   
    DESC: 显示filename中含有James字符串的下一行

 

14. SED q: quit
    $sed '100q' filename
    DESC: print the first 100 lines from filename
   
15. SED y: transform
    /.*/y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/
    DESC:replace all the characters into capital   
   
16. convert file 'filename' from 1       to 2
                          2     1
                          11     22
                          22         11
                          111        222
                          222        111
    $ sed '/1/{
    > h
    > d
    > }
    > /2/{
    > G
    > }' filename      
    h/H: Hold, copy or append contents of pattern space to hold space
    g/G: Get, copy or append contents of hold space to pattern space
    x: Exchange, swap contents of hold space and pattern space
   
17. AWK中的特殊字符
    a alert character, usually ASCII BEL character
    b backspace
    f formfeed
    n newline
    r carriage return
    t horizontal tab
    v vertical tab
    c any literal character c

 

18. AWD系统变量
    FS field separator
    NR the number of current input record
    FILENAME the name of current input file
    RS the record separator, as a newline
    NF the number of fields for the current record

 

    EXAMPLE awkcmd 列出各纪录平均值，并标出行号，最后显示文件名和记录数:
    #list the avg scores and row numbers         
    BEGIN { FS=" " }
    { print NR,".",$1,",",($2+$3+$4)/3 }
    END { print 
          print "File " FILENAME " has " NR " rows " }
   
    $awk -f awkcmd sourcefile

 

19. #将多行的块纪录转成单行纪录
    BEGIN { FS="n"; RS="" }
    { print $1, $NF }
   
20. 显示当前目录下各文件名和大小的例子   
    ls -lR|awk '
    BEGIN {print "        size","  ", "filename"}
    NF==9 && /^-/ {     #9列并以-开头表示文件
            filesize += $5
            ++filenum
            print " file ", $5,"  ",$9
            }
    NF==9 && /^d/ { #9列并以d开头表示子目录
            print "

", "t", $9
            }
    $1 ~ /^..*:$/ { # ~: Match Operator
       #ls -lR中子目录显示为 ./subdir:  以"."开头"$"结尾的行判断为子目录的行
            print "t" $0
            }
    END { print "Total: ", filesize, " bytes (", filenum, " files)"} '
   
21. AWK传递参数和检索数据
    $ cat awk_par
    awk '$2==search' search=$1 srcfile
    运行:
    $ awk_par par_val
    DESC: 参数par_val传递给变量search，显示srcfile中第二列为par_val的记录行
   
22. AWK中将shell参数传递给awk
    search=$1
    awk '$2 ~ /'"${search}"'/'  srcfile
    DESC: 将shell的第一个参数传递给变量search，同srcfile中第二列比较
       如果改成 awk '$2 ~ /'"${search:-.*}"'/' srcfile，则如果shell运行时无参数$1,就显示所有记录