wget下载目录/wget下载并覆盖原文件 bash基础

wget下载文件的时候，-N参数只是比较了时间和大小
我用的是-r参数，没有用-m，我发现-m 会在每个文件夹下生成.listing文件，而且如果我执行第二次镜像命令后，wget一直向log里面写数据，根本没有下载。。
如果本地和ftp同步一次后，ftp上一个文件被修改了。修改后的文件尺寸比本地的小，
那么wget 不下载这个文件。
如何让wget能覆盖本地文件呢？
以下是我尝试的方法，都没有解决覆盖本地已经存在的文件的问题。
wget -o D:\MyDocuments\test\log -nH -r -c -N -l inf -P D:\MyDocuments\test\source ftp://l:1@192.168.1.4:21/"visual studio"
wget -o D:\MyDocuments\test\log -nH -r -c -l inf -P D:\MyDocuments\test\source ftp://l:1@192.168.1.4:21/"visual studio"
wget -o D:\MyDocuments\test\log -nH -m -c -P D:\MyDocuments\test\source ftp://l:1@192.168.1.4:21/"visual studio"
wget -q -nH -m -c -P D:\MyDocuments\test\source ftp://l:1@192.168.1.4:21/"visual studio"
答案:
去掉 -c参数(该参数是续传...我以为续传是断了链接后才续传,没有想到wget不是我理解的那样,2楼正确)

 

代码:

wget -P . -m -nH ftp://user:password@ip/test

我测试了上面的语法是可以正常的。
你不应该用-c参数吧，-c是续传，尺寸小自然就不会覆盖了

 

linux SA经常使用，手动下载，或者放在脚本里面如下：
-m ftp://192.168.0.170/* –ftp-user=wugk –ftp-password=1112233 下载整个目录加m参数
如果想重新以前的文件如下：
-c ftp://192.168.0.170/* –ftp-user=wugk –ftp-password=1112233 只能下载文件，不能。
将放到这里备用：
下载有索引目录
-m http: // 这个最强力了！前提是目录必须是索引目录！

下载一个目录，例如网站的yourdir

-U “Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; GTB5)” -r -p -k -np -Pmydir -nc -o down.log

如果要想下载整个网站，最好去除-np参数。

-U “Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; GTB5)” -r -p -k -nc -o down.log

-U 修改agent，伪装成IE货firefox等
-r 递归；对于HTTP主机，首先下载URL指定的文件，然后（如果该文件是一个HTML文档的话）递归下载该文件所引用（超级连接）的所有文件（递归深度由参数-l指定）。对FTP主机，该参数意味着要下载URL指定的目录中的所有文件，递归方法与HTTP主机类似。
-c 指定断点续传功能。实际上，wget默认具有断点续传功能，只有当你使用别的ftp工具下载了某一文件的一部分，并希望wget接着完成此工作的时候，才需要指定此参数。
-nc 不下载已经存在的文件
-np 表示不跟随链接，只下载指定目录及子目录里的东西；
-p 下载页面显示所需的所有文件。比如页面中包含了图片，但是图片并不在/yourdir目录中，而在/images目录下，有此参数，图片依然会被正常下载。

-k 修复下载文件中的绝对连接为相对连接，这样方便本地阅读。

 

 

$ tree -d .

$ file tux_small.png
tux_small.png: PNG image data, 128 x 151, 8-bit/color RGB, non-interlaced

$ wc < RightNow.txt
2 12 58

$ cat RightNow.txt | wc
2 12 58

$ test -e . && echo "Yes."
Yes.

$ echo $(( 0 && 0 ))
0

$ echo $(( 1 && 0 ))
0

$ echo $(( 0 && 1 ))
0

$ echo $(( 1 && 1 ))
1

ls -1 | while read fn; do echo "$fn" done

 

n=1 while [ $n -le 6 ]; do echo $n let n++ done

$ ./myscript.sh
1
2
3
4
5
6

 

y=1 while [ $y -le 12 ]; do x=1 while [ $x -le 12 ]; do printf "% 4d" $(( $x * $y )) let x++ done echo "" let y++ done

$ ./myscript.sh

1 2 3 4 5 6 7 8 9 10 11 12 2 4 6 8 10 12 14 16 18 20 22 24 3 6 9 12 15 18 21 24 27 30 33 36 4 8 12 16 20 24 28 32 36 40 44 48 5 10 15 20 25 30 35 40 45 50 55 60 6 12 18 24 30 36 42 48 54 60 66 72 7 14 21 28 35 42 49 56 63 70 77 84 8 16 24 32 40 48 56 64 72 80 88 96 9 18 27 36 45 54 63 72 81 90 99 108 10 20 30 40 50 60 70 80 90 100 110 120 11 22 33 44 55 66 77 88 99 110 121 132 12 24 36 48 60 72 84 96 108 120 132 144

 

An advanced example with numbers and user input

• Here is an example guessing game that ties together some of the elements we've covered so far: secretNumber=$(( ((`date +%N` / 1000) % 100) +1 )) guess=-1 while [ "$guess" != "$secretNumber" ]; do echo -n "I am thinking of a number between 1 and 100. Enter your guess:" read guess if [ "$guess" = "" ]; then echo "Please enter a number." elif [ "$guess" = "$secretNumber" ]; then echo -e "\aYes! $guess is the correct answer!" elif [ "$secretNumber" -gt "$guess" ]; then echo "The secret number is larger than your guess. Try again." else echo "The secret number is smaller than your guess. Try again." fi done

  Please study this example carefully, and refer to the reference materials in to understand some of the methods.

Creating and using arrays

• Shell arrays are relatively easy to construct. Example: array=(red green blue yellow magenta) len=${#array[*]} echo "The array has $len members. They are:" i=0 while [ $i -lt $len ]; do echo "$i: ${array[$i]}" let i++ done

  Run this example:

  $ ./myscript.sh

  The array has 5 members. They are: 0: red 1: green 2: blue 3: yellow 4: magenta

  Now, before you decide this is a silly, rather useless example, replace one line of the script and run it again:

  array=(`ls`)

  See what difference this makes (and think of all the kinds of lists you might create for this line).

Strings and substrings

• It's useful to be able to take strings apart and put them together in different ways. Here is how to select a substring from a string: string="this is a substring test" substring=${string:10:9} In this example, the variable "substring" contains the word "substring". Remember this rule: substring=${string_variable_name:starting_position:length}

  The string starting position is zero-based.

Searching and Replacing Substrings within Strings

• In this method you can replace one or more instances of a string with another string. Here is the basic syntax: alpha="This is a test string in which the word \"test\" is replaced." beta="${alpha/test/replace}" The string "beta" now contains an edited version of the original string in which the first case of the word "test" has been replaced by "replace". To replace all cases, not just the first, use this syntax: beta="${alpha//test/replace}" Note the double "//" symbol.

  Here is an example in which we replace one string with another in a multi-line block of text:

  list="cricket frog cat dog" poem="I wanna be a x\n\ A x is what I'd love to be\n\ If I became a x\n\ How happy I would be.\n" for critter in $list; do echo -e ${poem//x/$critter} done Run this example:

  $ ./myscript.sh

  I wanna be a cricket A cricket is what I'd love to be If I became a cricket How happy I would be. I wanna be a frog A frog is what I'd love to be If I became a frog How happy I would be. I wanna be a cat A cat is what I'd love to be If I became a cat How happy I would be. I wanna be a dog A dog is what I'd love to be If I became a dog How happy I would be.

  Silly example, huh? It should be obvious that this search & replace capability could have many more useful purposes.

More obscure but useful string operations

• Here is a way to isolate something useful from a large, even multi-line, string. As above, this method relies on enclosing a variable name in curly braces, then aplying a special operator to achieve a particular result.

  Here is a list of four such operators:

  • Operator "#" means "delete from the left, to the first case of what follows."

    $ x="This is my test string." $ echo ${x#* } is my test string.

  • Operator "##" means "delete from the left, to the last case of what follows."

    $ x="This is my test string." $ echo ${x##* } string.

  • Operator "%" means "delete from the right, to the first case of what follows."

    $ x="This is my test string." $ echo ${x% *} This is my test

  • Operator "%%" means "delete from the right, to the last case of what follows."

    $ x="This is my test string." $ echo ${x%% *} This

  I find these operators particularly useful in parsing multi-line strings. Let's say I want to isolate a particular IP address from the output of the "ifconfig" command. Here's how I would proceed:

  $ x=`/sbin/ifconfig` $ echo $x eth0 Link encap:Ethernet HWaddr 00:0D:56:0C:8D:10 inet addr:192.168.0.1 Bcast:192.168.0.255 Mask:255.255.255.0 inet6 addr: fe80::20d:56ff:fe0c:8d10/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:253339 errors:0 dropped:0 overruns:0 frame:0 TX packets:423729 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:36150085 (34.4 MiB) TX bytes:496768499 (473.7 MiB) Base address:0xecc0 Memory:fe4e0000-fe500000 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:109394 errors:0 dropped:0 overruns:0 frame:0 TX packets:109394 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:12372380 (11.7 MiB) TX bytes:12372380 (11.7 MiB)

  There's lots of information, more than we need. Let's say for the sake of argument that I want the IP of "lo", the loopback interface. I could specify this:

  $ y=${x#*inet addr:}

  But, while gobbling text from the left, this search would stop at the IP address of eth0, not the desired interface. So I can specify it this way:

  $ y=${x#*lo *inet addr:}

  As a last step I'll trim off all remaining text to the right:

  $ y=${y%% *}

  Leaving only the desired address.

  It seems the "#" and "%" operators, and their variants, are able to accept a rather complex argument and sort through the content of large strings, including strings with line breaks. This means I can use the shell to directly filter content in some simple cases where I might have considered using sed or Perl.

Bash Version 3

I have always thought the inability to test for the presence of a string or pattern (without using grep, sed or something similar) was a conspicuous weakness in shell programming. Bash version 3, present on must current Linux distributions, addresses this lack by allowing regular expression matching.

Let's say we need to establish whether variable $x appears to be a social security number:

if [[ $x =~ [0-9]{3}-[0-9]{2}-[0-9]{4} ]] then # process SSN else # print error message fi

Notice the Perlish "=~" syntax and that the regular expression appears within double brackets. A substantial number of regular expression metacharacters are supported, but not some of the Perl-specific extensions like \w, \d and \s.

Another Bash 3 feature is an improved brace expansion operator:

$ echo {a..z} a b c d e f g h i j k l m n o p q r s t u v w x y z

for n in {0..5} do echo $n done 0 1 2 3 4 5

Useful Links

Well, as long-winded as it turned out to be, this page is supposed to be an introduction to shell programming, one that just touches the highlights. The linked references below will provide a deeper understanding of the covered topics.

• Advanced Bash Scripting Guide (Linux Documentation Project)