使用AWSTATS自动分析Nginx日志

环境说明：

Nginx配置文件存放目录：/ect/nginx/cnginx.conf

Nginx日志目录：/var/log/nginx/

Awstats安装目录：/usr/local/awstats

Awstats配置文件存放目录：/etc/awstats

日志切割脚本存放目录：/usr/local/nginx/nginx_log.sh

一、Nginx日志切割

1、修改nginx.conf配置文件，使AWSTATS支持分析日志格式

# vim /usr/local/nginx/conf/nginx.conf      //去掉前面的#号

log_format main '$remote_addr  - $remote_user [$time_local] "$request" '

              '$status $body_bytes_sent  "$http_referer" '

              '"$http_user_agent"  "$http_x_forwarded_for"';

access_log logs/access.log main;

# vim /usr/local/nginx/nginx_log.sh

#!/bin/bash

# This script run at 00:00

mv   /usr/local/nginx/logs/access.log /usr/local/nginx/logs/access_$(date  -d "yesterday" +"%Y%m%d").log

#find /usr/local/nginx/logs/access_$(date-d  "yesterday" +"%Y%m%d").log -mtime +20 -exec rm -rf {} \;

kill -USR1 `cat  /usr/local/nginx/logs/nginx.pid`

# chmod +x /usr/local/nginx/nginx_log.sh

三、下载安装Awstats

1、下载安装

# wget  

# tar xvf awstats-7.0.tar.gz

# mv awstats-7.0 /usr/local/awstats

# chown –R root:root /usr/local/awstats

2、配置

# cd /usr/local/awstats/tools

# ./awstats_configure.pl

----- AWStats awstats_configure 1.0  (build 1.9) (c) Laurent Destailleur -----

This tool will help you to configure  AWStats to analyze statistics for

one web server. You can try to use it  to let it do all that is possible

in AWStats setup, however following  the step by step manual setup

documentation (docs/index.html) is  often a better idea. Above all if:

- You are not an administrator user,

- You want to analyze downloaded log  files without web server,

- You want to analyze mail or ftp log  files instead of web log files,

- You need to analyze load balanced  servers log files,

- You want to 'understand' all  possible ways to use AWStats...

Read the AWStats documentation (docs/index.html).

-----> Running OS detected: Linux,  BSD or Unix

-----> Check for web server install

Enter full config file path of your  Web server.

Example: /etc/httpd/httpd.conf

Example:  /usr/local/apache2/conf/httpd.conf

Example: c:\Program files\apache  group\apache\conf\httpd.conf

Config file path ('none' to skip web  server setup):

> none #这里不是apache，写none，跳过#

回车

Your web server config file(s) could  not be found.

You will need to setup your web server  manually to declare AWStats

script as a CGI, if you want to build  reports dynamically.

See AWStats setup documentation (file  docs/index.html)

-----> Update model config file  '/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf'

  File awstats.model.conf updated.

-----> Need to create a new config  file ?

Do you want me to build a new AWStats  config/profile

file (required if first install) [y/N]  ?输入y #创建一个全新的统计

回车

-----> Define config file name to  create

What is the name of your web site or  profile analysis ?

Example:

Example: demo

Your web site, virtual server or  profile name:

> benet   #被统计网站的名字，随便起 #

回车

-----> Define config file path

In which directory do you plan to  store your config file(s) ?

Default: /etc/awstats

Directory path to store config file(s)  (Enter for default):

> #回车

-----> Create config file  '/etc/awstats/awstats.benet.conf'

Config file /etc/awstats/awstats.benet.conf  created.

-----> Add update process inside a  scheduler

Sorry, configure.pl does not support  automatic add to cron yet.

You can do it manually by adding the  following command to your cron:

/usr/local/awstats/wwwroot/cgi-bin/awstats.pl  -update -config=benet

Or if you have several config files  and prefer having only one command:

/usr/local/awstats/tools/awstats_updateall.pl  now

Press ENTER to continue...   #回车

A SIMPLE config file has been created:  /etc/awstats/awstats.benet.conf

You should have a look inside to check  and change manually main parameters.

You can then manually update your  statistics for 'benet' with command:

> perl awstats.pl -update -config=benet

You can also build static report pages  for 'benet' with command:

> perl awstats.pl -output=pagetype  -config=benet

Press ENTER to finish...   #回车结束

四、修改统计日志文件的路径

# vim/etc/awstats/awstats.benet.conf

LogFile="/var/log/httpd/mylog.log"修改成--->

LogFile="/etc/nginx/access_%YYYY-0%MM-0%DD-24.log"

这里是对应上面Nginx日志切割所生成的目录存放位置，注意awstats的年月日格式，分析的执行顺序是：

Nginx 产生日志 –> 日志切割 –> Nginx 继续产生日志 –> 另存切割日志 –> 交由Awstats统计 –> 生成结果

五、执行Awstats 日志更新程序开始统计分析

# mkdir -p /var/lib/awstats

# /usr/local/awstats/wwwroot/cgi-bin/awstats.pl  -update -config=benet

Create/Update database for config  "/etc/awstats/awstats.benet.conf" by AWStats version 7.0 (build  1.971)

From data in log file  "/usr/local/nginx/logs/access_20131108.log"...

Error: Couldn't open server log file  "/usr/local/nginx/logs/access_20131108.log" : 没有那个文件或目录

上面的报错是说没有找到你指定的那个文件目录，手动创建一个

#  touch  /etc/nginx/access_20131108.log

# /usr/local/awstats/wwwroot/cgi-bin/awstats.pl  -update -config=benet

Create/Update database for config  "/etc/awstats/awstats.benet.conf" by AWStats version 7.0 (build  1.971)

From data in log file  "/usr/local/nginx/logs/access_20131108.log"...

Phase 1 : First bypass old records,  searching new record...

Searching new records from beginning  of log file...

Jumped lines in file: 0

Parsed lines in file: 0

Found 0 dropped records,

Found 0 comments,

Found 0 blank records,

Found 0 corrupted records,

Found 0 old records,

Found 0 new qualified records.

看到以上显示，证明日志切割和 Awstats 都已经运行无误了。统计分析完成后，结果还在 Awstats 的数据库中。在 Apache 上，可以直接打开 Perl 程序的网页查看统计。但本文开始时已经提到，Nginx 对 Perl 支持并不好，所以我们要换个方法，利用awstats 的工具将统计的结果生成静态文件。

六、Awstats生成静态文件

# mkdir –p /www/site/awstats

# /usr/local/awstats/tools/awstats_buildstaticpages.pl  -update  -config=benet -lang=cn -dir=/www/site/awstats  -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl

/usr/local/awstats/tools/awstats_buildstaticpages.pl #Awstats 静态页面生成脚本

-update -config=benet #更新配置选项

-lang=cn #语音中文

-dir=/var/www/awstats/ #统计结果输出目录

-awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl#日志更新程序路径

七、修改nginx配置文件

# vim/etc/nginx/nginx.conf

在文件末尾添加如下行

server {

   listen       8080;

   server_name  localhost;

   root   /www/site/awstats;

   index  index.html index.htm index.php default.html default.htm default.php;    

autoindex on;      #开启目录浏览功能

   error_log  off;

   charset  gb2312;

   location  ~ ^/awstats/ {     # html 静态页面目录

        root   /www/site/awstats;

        index  index.html;

        autoindex on;

        access_log off;

        error_log off;

        charset gb2312;  

        }

        location ~ ^/icon/ {              # 图标目录

        root    /usr/local/awstats/wwwroot;

        index  index.html;

        autoindex on;

        access_log off;

        error_log off;

        charset gb2312;

        }

}

八、结果验证，URL输入

     #目录结构形式浏览

      #总结果形式浏览,如果想看总浏览，就在autoindex on前加#号，在每个index后加awstats.benet.html

九、定时任务计划

# crontab -e

01 00 * * * /bin/bash  /usr/local/nginx/nginx_log.sh

30 00 * * * /usr/local/awstats/wwwroot/cgi-bin/awstats.pl  -update -config=benet

00 03 * * */usr/local/awstats/tools/awstats_buildstaticpages.pl  -update  -config=benet -lang=cn -dir=/www/site/awstats   -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl

十、保护日志统计结果页面

一般站长都不愿随便让人知道自己站的真实流量，所以要把 Awstats 统计结果页面进行密码保护。Nginx 使用的是跟 Apache 一样的密码加密格式，这里需要用到 apache 自带的工具htpasswd。

如果你在本机上默认装有 Apache，这你就只需在它的程序目录下运行

例:

#/usr/local/apache2/bin/htpasswd -c  admin.pass admin #用户名为admin

New password:           输入密码

Re-type new password:       重复输入

Adding password for user admin  创建成功

然后把 admin.pass 这个密码包找个的地方藏起来.

修改 nginx.conf 在 location 中加入(红色加粗部分)：

server {

   listen       8080;

   server_name  localhost;

   root   /www/site/awstats;

   index  index.html index.htm index.php default.html default.htm default.php;    

autoindex on;

   error_log  off;

   charset  gb2312;

   location  ~ ^/awstats/ {     # html 静态页面目录

        root   /www/site/awstats;

        index  index.html;

        autoindex on;

        access_log off;

        error_log off;

        charset gb2312;  

auth_basic     "admin"; #用户名

     /opt/ngx/conf/admin.pass; #密码包路径

        }

        location ~ ^/icon/ {              # 图标目录

        root    /usr/local/awstats/wwwroot;

        index  index.html;

        autoindex on;

        access_log off;

        error_log off;

        charset gb2312;

        }

}

修改 Nginx 配置完毕后，执行命令  killall –s HUP nginx 让 Nginx 重新加载配置即可。

<注：>该文章摘自：http://lxw66.blog.51cto.com/5547576/1353890
            其中nginx的安装目录与我的安装目录不一样，我在该文章里做了相应的修改；
            其中的一、二、八、九、十我都没有做，一二是因为不明确要做什么，八九十也有一定的错误，还望大家来解答！！