Linux Shell Scripting Cookbook

Linux Shell Scripting Cookbook 笔记


变量
内部变量
$SHELL        当前系统shell，比如/bin/bash
$UID            root用户UID为零， ubuntu中sudo sh script.sh
$RANDOM        返回一个随机数
$IFS            shell默认分隔符，默认空格， 可以设置 IFS=”:”
$PATH            系统查找路径
$PWD            当前目录
$HOME        HOME目录
$$            当前进程ID
$#            参数个数
$?            脚本文件或函数返回值
$0            脚本文件或函数名
$1            脚本文件或函数第一个参数， 以此类推, shift可以对参数移位
$*             同$@， 表示所有参数
“$*”            “$1$2$3...”
“$@”            “$1” “$2” “$3” …

[[:alnum:]]         数字字符
[[:digit:]]         数字字符   
[[:xdigit:]]         十六进制字符
[[:alpha:]]         字母字符
[[:lower:]]        小写字母字符
[[:upper:]]         大写字母字符
[[:punct:]]         标点符号字符
[[:graph:]]         非空格字符   
[[:space:]]         空白字符
[[:blank:]]         空格和tab字符
[[:cntrl:]]         控制字符
[[:print:]]         可显示字符

变量操作
$ variable    获取变量值
${variable}    获取变量值
${#variable}     获取变量长度
${#variable:=word}     变量存在且不为NULL，返回变量，否则设置变量为word并返回word
${#variable:?message} 变量存在且不为NULL，返回变量，否则输出message并退出
${#variable:+word} 变量存在且不为NULL，返回word，否则返回NULL
${variable%.*}    从右往左，匹配变量中.*, 并删除该部分（获取扩展名外文件名部分）
${variable%%.*}    从右往左，最大程序匹配变量中.*, 并删除该部分（只留第一个点号左边部分）
${variable#*.}    从左往右，匹配变量中*., 并删除该部分（保留第一个点号右边部分）
${variable##*.}      从左往右，最大程序匹配变量中*., 并删除该部分（获取文件名扩展名部分）

变量计算
let result=no1+no2
result=$[ $no1 + $no2 ]
result=$( ($no1 + $no2) )
result=`expr 3+4`
result=$( expr $no1 + 5 )

数组
array_var1=("test1" "test2" "test3")
array_var2=(TEST1 TEST2 TEST3)
echo array_var1[1]=${array_var1[1]}
index=2
echo array_var2[2]=${array_var1[$index]}
echo array_var1: ${array_var1[*]}
echo array_var2: ${array_var1[@]}
echo array_var1 len: ${#array_var1[*]}
echo array_var2 index: ${!array_var2[*]}

$ declare -A fruits_value
$ fruits_value=([apple]='100dollars' [orange]='150 dollars')
$ echo "Apple costs ${fruits_value[apple]}"
Apple costs 100 dollars
$ echo ${!array_var[*]} #show all the index in associative arrays
orange apple

条件判断
Unary operators that check file characteristics Option Description
-b     File is block special device (for files like /dev/hda1)
-c     File is character special (for files like /dev/tty)
-d     File is a directory
-e     File exists
-f     File is a regular file
-g     File has its set-group-ID bit set
-h     File is a symbolic link (same as -L)
-G     File is owned by the effective group ID
-k     File has its sticky bit set
-L     File is a symbolic link (same as -h)
-O     File is owned by the effective user ID
-p     File is a named pipe
-r     File is readable
-s     File has a size greater than zero
-S     File is a socket
-u     File has its set-user-ID bit set
-w     File is writable
-x     File is executable

-lt     <         Less than
-le     <=         Less than or equal to
-gt     >         Greater than
-ge     >=         Greater than or equal to
-eq     =, ==         Equal to
-ne     !=         Not equal to

if [ -r $FILE -a -w $FILE ]
if [ -z "$V1" -o -z "${V2:=YIKES}" ]
if [[ "${MYFILENAME}" == *.jpg ]]
if [[ "$FN" == *.@(jpg|jpeg) ]]
if [[ "$CDTRACK" =~ "([[:alpha:][:blank:]]*)- ([[:digit:]]*) - (.*)$" ]]


@( ... ) Only one occurrence
*( ... ) Zero or more occurrences
+( ... ) One or more occurrences
?( ... ) Zero or one occurrences
!( ... ) Not these occurrences, but anything else

echo相关问题

单引号、双引号、不加引号的区别
$ echo "cannot include exclamation - ! within double quotes"
This will return the following:
bash: !: event not found error

双引号内需要对特殊字符转义。
如果不加引号， 分号会被看成命令分隔符号
单引号内的变量不会被赋值，单引号可以

echo选项
-e    使用转义，echo -e "1\t2\t3"
-n    忽略结尾换行

彩色显示
Color codes are used to represent each color. For example, reset=0, black=30, red=31,
green=32, yellow=33, blue=34, magenta（洋红）=35, cyan（青色）=36, and white=37.

For a colored background, reset = 0, black = 40, red = 41, green = 42, yellow = 43, blue = 44,
magenta = 45, cyan = 46, and white=47, are the color code that are commonly used.

echo -e "\e[1;31m This is red text \e[0m"
echo -e "\e[1;42m Green Background \e[0m"
echo -e "\e[1;42m \e[1;31m This is Green Background, red text \e[0m"

终端信息
Get number of columns and rows in a terminal as follows:
    tput cols
    tput lines
In order to print the current terminal name, use:
    tput longname
For moving the cursor to a position 100,100 you can enter:
    tput cup 100 100
Set the background color for terminal as follows:
    tput setb no     #no can be a value in the range of 0 to 7.
Set the foreground color for text as follows:
    tput setf no     #no can be a value in the range of 0 to 7.
In order to make the text bold use:
    tput bold
Start and end underlining by using:
    tput smul
    tput rmul
In order to delete from cursor to end of the line use:
    tput ed

The -echo option above disables output to the terminal, whereas echo enables output.
    stty -echo
    read password
    stty echo

调试
set -x: Displays arguments and commands upon their execution
set +x: Disables debugging
set –v: Displays input when they are read
set +v: Disables printing input

xargs技巧
Converting multiple lines of input to a single line output:
or Converting single line into multiple line output:

$ cat example.txt # Example file
1 2 3 4 5 6
7 8 9 10
11 12
$ cat example.txt | xargs
1 2 3 4 5 6 7 8 9 10 11 12
$ cat example.txt | xargs -n 3
1 2 3
4 5 6
7 8 9
10 11 12

We can also use our own delimiter towards separating arguments.
$ echo "splitXsplitXsplitXsplit" | xargs -d X
split split split split
$ echo "splitXsplitXsplitXsplit" | xargs -d X -n 2
split split
split split

To provide a command execution sequence as shown, xargs has an option –I.
$ cat args.txt | xargs -I {} ./cecho.sh -p {} -l

xargs and find are best friends.
$ find . -type f -name "*.txt" -print0 | xargs -0 rm -f
$ find source_code_dir_path -type f -name "*.c" -print0 | xargs -0 wc -l


$ cmd0 | ( cmd1;cmd2;cmd3) | cmd4
If cmd1 is cd /, within the subshell, the path of the working directory changes. However, this
change resides inside the subshell only. cmd4 will not see the directory change.


重定向
echo "This is a sample text 1" > temp.txt
echo "This is sample text 2" >> temp.txt
command >stderr.txt 1>stdout.txt
command >&1 output.txt
command | tee FILE | command2 将command的输出存入文件FILE，同时给command的输入
command > /dev/null
cat <log.txt

命令交互
Let's write a script that reads input interactively and uses this script for automation examples:

#!/bin/bash
#Filename: interactive.sh
read -p "Enter number:" no ;
read -p "Enter name:" name
echo You have entered $no, $name;

方式一：
$ ./interactive.sh
Enter number:1
Enter name: hello
You have entered 1, hello


方式二：
$ echo -e "1\nhello\n" | ./interactive.sh
You have entered 1, hello


方式三：
$ echo -e "1\nhello\n"
> input.data
$ ./interactive.sh < input.data
You have entered 1, hello


方式四：
#!/usr/bin/expect
#Filename: automate_expect.sh
spawn ./interactive .sh
expect "Enter number:"
send "1\n"
expect "Enter name:"
send "hello\n"
expect eof


$ ./automate_expect.sh

read "n" characters from input :
    $ read -n 8 var
Read a password in non-echoed mode as follows:
    $ read -s var
Display a message with read using:
    $ read -p "Enter input:" var
Read the input after a timeout :
    $ read -t 2 var
Use a delimiter character to end the input line as follows:
    read -d “:” var        #读取到var, “:”作为结束分隔符

文件重命名
将所有jpg，png文件重新排序命名为image-*.jpg 或image-*.png
#!/bin/bash

#Filename: rename.sh
#Description: Rename jpg and png files
count=1;
for img in *.jpg *.png
do
new=image-$count.${img##*.}
mv "$img" "$new" 2> /dev/null
if [ $? -eq 0 ];
then
echo "Renaming $img to $new"
let count++
fi
done

将所有JPG文件命名为jpg
rename *.JPG *.jpg

将所有文件或文件夹名中的空白变成'_'
rename 's/ /-/g' *

转换所有文件或文件夹名大小写
rename 'y/A-Z/a-z/' *
rename 'y/A-Z/a-z/' *

将所有txt文件名中的空白变成'_'
rename 's/ /-/g' *.txt
find ./ -type f -name *.txt -exec rename 's/ /-/g' {} \;



文件查找
$ find /home/slynux -name "*.txt" –print
$ find . -iname "example*" -print             #-iname和-name 一样，只是忽略大小写
$ find /home/users -path "*slynux*" -print         #-path会匹配整个路径，不止是文件名
$ find . \( -name "*.txt" -o -name "*.pdf" \) -print     #多个条件中的一个
$ find . -regex ".*\(\.py\|\.sh\)$"
$ find . -iregex ".*\(\.py\|\.sh\)$"
$ find . ! -name "*.txt" -print                 #！表示取反， 查找txt以外的文件
$ find . -maxdepth 1 -type f -print
$ find . -mindepth 2 -type f -print
$ find . -type d -print                     #type 有d, f, l,p,c,b,s
$ find . -type f -atime -7 -print             #7天内访问过的文件           
$ find . -type f -atime 7 -print                 #刚好7天前那天访问过的文件
$ find . -type f -atime +7 -print             #7天前访问过的文件
$ find . -type f -newer file.txt -print             #mtime 大于file .txt (比之更新的)的文件
$ find . -type f -size +2k                 #大小大于2K 的文件（c,w,k,M,G）
$ find . –type f –name "*.php" ! -perm 644 –print     #根据权限查找
$ find . -type f -user slynux -print             #根据用户查找
$ find . -type f -name "*.swp" -delete         #找到后删除文件
$ find . -type f -name "*.c" -exec cat {} \;>all_c_files.txt         #{} 匹配找到的文件
$ find . -type f -mtime +10 -name "*.txt" -exec cp {} OLD \;    
$ find . -type f -name "*.txt" -exec printf "Text file: %s\n" {} \;
$ find devel/source_path \( -name ".git" -prune \) -o \( -type f -print \) #-prune 跳过.git文件夹

-atime: This is the last timestamp of when the file was accessed by some user
-mtime: This is the last timestamp of when the file content was modified
-ctime: This is the last timestamp of w hen the file permissions or ownership was modified

文件内容查找
$ grep "match_text" file1 file2 file3 ...
$ grep word filename –-color=auto             #高亮匹配部分
$ egrep "[a-z]+"
$ grep -v match_pattern file                 #显示不匹配的行的内容
$ grep -c "text" filename                 #显示匹配行数
$ echo -e "1 2 3 4\nhello\n5 6" | egrep -o "[0-9]" | wc -l     #显示匹配的次数
$ grep linux -n sample1.txt                 #显示匹配的行行号
$ echo gnu is not unix | grep -b -o "not"         #显示字节偏移
$ grep -l linux sample1.txt sample2.txt         #显示匹配的文件名 -L显示不匹配的文件名
$ grep "text" . -R -n                     #递归查找单前目录
$ echo hello world | grep -i "HELLO"         #忽略大小写
$ echo this is a line of text | grep -e "this" -e "line" -o #匹配多个字符串中的一个
$ grep "main()" . -r --include *.{c,cpp}
$ grep "main()" . -r –-exclude "README"
$ grep "test" file* -lZ | xargs -0 rm

$ sed 's/PATTEN/replace_text/' filename         #查找并替换， 输出到终端
$ sed 's/PATTERN/replacement/' -i filename     #-i 直接修改文件


常用命令
tempfile        生存临时文件
md5sum sha1sum     文件校验
sort            排序
uniq            查看单一或重复
tr            字符删除，替换
sed            文档、字符串处理   
cut            字段裁剪 cut -d : -f 6 /etc/password :分隔字段，输出第六个字段内容。
find            文件查找
date            显示日期
sleep            延时多少秒
tree            打印目录树
wc            行数，单词、字符数统计
head/tail        显示开头/结尾的多少行
sed
awk