sed (Stream Editor) grep (global regular expression print) awk

sed 's/cat/dog/g' pets

sed 's/cat/dog/gi' pets

sed '1,50s/dog/cat/g' pets

sed '/digby/,/duncan/s/dog/cat/g' pets
使用字符串搜索来指定地址，从包含'digby' 的那一行开始到 包含'duncan'的那一行为止，把 dog 改为 cat

多个 sed 指令
sed -e 's/dog/cat/' -e 's/hi/lo/' pets
要进行大量编辑，将命令保存的文件，使用 -f 选项调用.
sed -f myedits pets

------------------------------------------------

只显示 /etc/ 目录中所有包含 root 这个词的文件列表，不要显示出错信息！
grep -l 'root' /etc/* 2>/dev/null

1.grep -w 'hello' file

2.echo "sayhello" | grep 'hello\>'

  echo "helloyo" | grep '\

3.grep 'bsd' /etc/passwd /dev/null

  grep -H 'bsd' /etc/passwd

4.grep -C 2 'hello' file

5.grep -rH --include='*.c' 'hello /home/cgi

  find /home/cgi -name '*.c' -print0 | xargs -0r grep -H 'hello'

7.ps aux |grep '[c]ron'

8.I can do OR with '|',but what about AND?

  grep 'paul' /etc/motd | grep 'franc,jacky'

  finds all lines that contain both 'paul' and 'franc,jacky'

9.How can I search in both standard input and in files?

  Use the special file name '-':

  cat /etc/passwd | grep 'alain' - /etc/motd

10.How to express palindromes in a regular expression?

   grep -W -e '\(\.)\(.\)\2\1' file

   It matches the word 'radar' of 'civic'..

   grep -E -e '^(.?)(.?)(.?)(.?)(.?)(.?)(.?)(.?)(.?).?\9\8\7\6\5\4\3\2\1$' file

\t a tab character
\n a newline character
\r a carriage-r eturn character
\s matches any ?whitespace? character (space, tab, newline, formfeed, and such)
\S anything not !\s"
\w ![a-zA-Z0-9R]" (useful as in !\w+", ostensibly to match a word)
\W anything not !\w", i.e., ![^a-zA-Z0-9R]"
\d ![0-9]", i.e., a digit
\D anything not !\d", i.e., ![^0-9]"

The essential organization of an AWK program follows the form:

pattern { action }

The pattern specifies when the action is performed. Like most UNIX utilities, AWK is line oriented. That is, the pattern specifies a test that is performed with each line read as input. If the condition is true, then the action is taken. The default pattern is something that matches every line. This is the blank or null pattern. Two other important patterns are specified by the keywords "BEGIN" and "END". As you might expect, these two words specify actions to be taken before any lines are read, and after the last line is read. The AWK program below:

BEGIN { print "START" }

      { print         }

END   { print "STOP"  }

adds one line before and one line after the input file. 

There are only a few commands in AWK. The list and syntax follows:

if ( conditional ) statement [ else statement ]

while ( conditional ) statement

for ( expression ; conditional ; expression ) statement

for ( variable in array ) statement

break

continue

{ [ statement ] ...}

variable=expression

print [ expression-list ] [ > expression ]

printf format [ , expression-list ] [ > expression ]

next 

exit

At this point, you can use AWK as a language for simple calculations; If you wanted to calculate something, and not read any lines for input, you could use the BEGIN keyword discussed earlier, combined with a exit command:

AWK去重复原理：

例如，!array[name]++ 

首先，new 一个数组，此时arrary[name] return False
然后，给arrary赋值，此时arrary[name] return True
最后，logical not

结果，
array[name]被赋值后，再次读入相同name时， 
arrary[name] return True ，action执行；
取反后，array[name] return False，action不执行。

Reference：