HI技术分享-Linux命令

说明

wget命令用于从网络上下载某些资源，该命令对于能够连接到互联网的Linux系统的作用非常大，可以直接从网络上下载自己所需要的文件。
wget的特点如下：
支持断点下载功能；

支持ftp额http下载方式；
支持代理服务器；
非常稳定，他在带宽很窄的情况下或不稳定的网络中都很强的适应性。如果由于网络原因下载失败，wget会不断的尝试，直到整个文件都下载完。如果服务打断了下载过程，他会再次连接到服务器上从停止的地方继续下载。这对那些从限定了连接时间的服务器上下载大文件非常有用。

格式

wget [option] [url]

常用参数

-o:将命令的执行结果写入到文件中
-O：指定保存的文件名后下载文件
—limit-rate：限速下载
-b：转入后台执行命令
-c：断点续传
—user-agent：指定客户端标志
-q：关闭下载时的输出
—tries=number：设置重试次数
—spider：模拟爬虫访问
-T seconds：设置访问超时时间
-P：指定下载文件的保存目录

示例

1、下载单个文件

[root@localhost ~]# wget http://download.redis.io/releases/redis-4.0.9.tar.gz
--2018-06-20 04:50:09--  http://download.redis.io/releases/redis-4.0.9.tar.gz
Resolving download.redis.io (download.redis.io)... 109.74.203.151
Connecting to download.redis.io (download.redis.io)|109.74.203.151|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1737022 (1.7M) [application/x-gzip]
Saving to: ‘redis-4.0.9.tar.gz’

100%[===================================================================================================================================================>] 1,737,022    296KB/s   in 6.2s    

2018-06-20 04:50:16 (275 KB/s) - ‘redis-4.0.9.tar.gz’ saved [1737022/1737022]

[root@localhost ~]#

2、使用-O选项指定下载文件的保存文件名

[root@localhost ~]# wget -O redis_new.tar.gz  http://download.redis.io/releases/redis-4.0.9.tar.gz
--2018-06-20 04:52:18--  http://download.redis.io/releases/redis-4.0.9.tar.gz
Resolving download.redis.io (download.redis.io)... 109.74.203.151
Connecting to download.redis.io (download.redis.io)|109.74.203.151|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1737022 (1.7M) [application/x-gzip]
Saving to: ‘redis_new.tar.gz’

100%[===================================================================================================================================================>] 1,737,022   42.8KB/s   in 26s    

2018-06-20 04:52:44 (66.0 KB/s) - ‘redis_new.tar.gz’ saved [1737022/1737022]

[root@localhost ~]# ll redis_new.tar.gz 
-rw-r--r--. 1 root root 1737022 Mar 26 12:04 redis_new.tar.gz

3、限速下载(—limit-rate)

[root@localhost ~]# wget --limit-rate=3k  http://download.redis.io/releases/redis-4.0.9.tar.gz
--2018-06-20 04:54:25--  http://download.redis.io/releases/redis-4.0.9.tar.gz
Resolving download.redis.io (download.redis.io)... 109.74.203.151
Connecting to download.redis.io (download.redis.io)|109.74.203.151|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1737022 (1.7M) [application/x-gzip]
Saving to: ‘redis-4.0.9.tar.gz.1’

 5% [======>                                                                                                                                             ] 87,291      3.00KB/s  eta 8m 57s

4、断点续传(-c)

[root@localhost ~]# wget -c  http://download.redis.io/releases/redis-4.0.9.tar.gz
--2018-06-20 04:56:10--  http://download.redis.io/releases/redis-4.0.9.tar.gz
Resolving download.redis.io (download.redis.io)... 109.74.203.151
Connecting to download.redis.io (download.redis.io)|109.74.203.151|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1737022 (1.7M) [application/x-gzip]
Saving to: ‘redis-4.0.9.tar.gz’

 8% [===========>                                                                                                                                        ] 148,987     57.6KB/s             ^C #强制终止
[root@localhost ~]# wget -c  http://download.redis.io/releases/redis-4.0.9.tar.gz #继续下载
--2018-06-20 04:56:16--  http://download.redis.io/releases/redis-4.0.9.tar.gz
Resolving download.redis.io (download.redis.io)... 109.74.203.151
Connecting to download.redis.io (download.redis.io)|109.74.203.151|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 1737022 (1.7M), 1567203 (1.5M) remaining [application/x-gzip]
Saving to: ‘redis-4.0.9.tar.gz’

20% [++++++++++++++===============>                                                                                                                      ] 359,852     47.7KB/s  eta 26s    ^C
[root@localhost ~]# wget -c  http://download.redis.io/releases/redis-4.0.9.tar.gz
--2018-06-20 04:56:22--  http://download.redis.io/releases/redis-4.0.9.tar.gz
Resolving download.redis.io (download.redis.io)... 109.74.203.151
Connecting to download.redis.io (download.redis.io)|109.74.203.151|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 1737022 (1.7M), 1356170 (1.3M) remaining [application/x-gzip]
Saving to: ‘redis-4.0.9.tar.gz’

25% [++++++++++++++++++++++++++++++++=====>                                                                                                              ] 446,845     43.2KB/s             ^C

5、使用wget -b后台下载文件

[root@localhost ~]# wget -b  http://download.redis.io/releases/redis-4.0.9.tar.gz
Continuing in background, pid 2641.
Output will be written to ‘wget-log’.
[root@localhost ~]# tail wget-log 
Saving to: ‘redis-4.0.9.tar.gz.2’

     0K .......... .......... .......... .......... ..........  2% 44.7K 37s
    50K .......... .......... .......... .......... ..........  5% 57.7K 32s
   100K .......... .......... .......... .......... ..........  8% 40.2K 33s
   150K .......... .......... .......... .......... .......... 11% 48.7K 32s
   200K .......... .......... .......... .......... .......... 14% 34.7K 33s
   250K .......... .......... .......... .......... .......... 17% 39.9K 32s
   300K .......... .......... .......... .......... .......... 20% 32.0K 33s

[root@localhost ~]# tail wget-log 
  1350K .......... .......... .......... .......... .......... 82% 58.1K 7s
  1400K .......... .......... .......... .......... .......... 85% 58.7K 6s
  1450K .......... .......... .......... .......... .......... 88% 35.5K 4s
  1500K .......... .......... .......... .......... .......... 91% 44.9K 3s
  1550K .......... .......... .......... .......... .......... 94% 55.7K 2s
  1600K .......... .......... .......... .......... .......... 97% 44.1K 1s
  1650K .......... .......... .......... .......... ......    100% 55.0K=38s

2018-06-20 04:58:37 (44.7 KB/s) - ‘redis-4.0.9.tar.gz.2’ saved [1737022/1737022]

6、伪装代理名称下载(—user-agent)

[root@localhost ~]# wget --user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36" http://download.redis.io/releases/redis-4.0.9.tar.gz
--2018-06-20 05:01:32--  http://download.redis.io/releases/redis-4.0.9.tar.gz
Resolving download.redis.io (download.redis.io)... 109.74.203.151
Connecting to download.redis.io (download.redis.io)|109.74.203.151|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1737022 (1.7M) [application/x-gzip]
Saving to: ‘redis-4.0.9.tar.gz.3’

100%[===================================================================================================================================================>] 1,737,022   43.1KB/s   in 25s    

2018-06-20 05:01:58 (67.0 KB/s) - ‘redis-4.0.9.tar.gz.3’ saved [1737022/1737022]

7、监控网站URL是否正常

[root@localhost ~]# wget -q -T 3 --tries=1 --spider www.hi917.com #采用静默访问方式，3秒超时，重试1次，模拟爬虫的方式进行访问
[root@localhost ~]# echo $?
0
#返回0表示正常

8、指定下载文件的保存目录

[root@localhost ~]# wget -P /tmp/ http://download.redis.io/releases/redis-4.0.9.tar.gz #这里我们将下载的文件保存到/tmp目录下
--2018-10-08 19:18:25--  http://download.redis.io/releases/redis-4.0.9.tar.gz
Resolving download.redis.io (download.redis.io)... 109.74.203.151
Connecting to download.redis.io (download.redis.io)|109.74.203.151|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1737022 (1.7M) [application/x-gzip]
Saving to: ‘/tmp/redis-4.0.9.tar.gz’ #这里就可以看出保存的位置

7.10、wget：命令行下载工具

说明

格式

常用参数

示例