nginx+keepalive测试

本测试是在两台OS环境为centos 5.5 64位系统上进行。
nginx版本：
/usr/local/nginx/sbin/nginx -v
nginx version: nginx/1.0.12
keepalive版本：
/usr/local/keepalived/sbin/keepalived -v
Keepalived v1.1.17 (11/07,2012)

 

 

一、安装nginx和keepalive

1、nginx的安装网上太多，这里忽略。

2、安装keepalive，两台测试机都要安装。
wget           ####下载包

tar zxvf keepalived-1.1.17.tar.gz                                         ####解压包

cd keepalived-1.1.17

./configure --prefix=/usr/local/keepalived                         ####编译安装并指定目录

make

make install

 

cp /usr/local/keepalived/sbin/keepalived /usr/sbin/

cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/

cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/

mkdir /etc/keepalived

cd /etc/keepalived

vim keepalived.conf                                    ####配置文件内容如下（主）
! Configuration File for keepalived

global_defs {
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state MASTER                                            ####主用MASTER，从用BACKUP
interface eth0                                          ####监测的网络接口
virtual_router_id 51                                    ####主从上要设置一样的值
#backup ipaddress
mcast_src_ip 10.1.1.191                                 ####主机的IP，备机更换备机用的IP
priority 100                                            ####权重，备机比主机稍微调小一点就行
advert_int 1                                            ####VRRP Multicast广播周期秒数
authentication {
auth_type PASS                                          ####VRRP认证方式
auth_pass qwerty                                        ####认证密码       
}
virtual_ipaddress {
10.1.1.190                                              ####VRRP
}
}

3、启动keepalive
/etc/init.d/keepalived start
启动后查看状态： ip addr | grep eth0
2: eth0: mtu 1500 qdisc pfifo_fast qlen 1000
    inet 10.1.1.191/24 brd 10.1.1.255 scope global eth0
    inet 10.1.1.190/32 scope global eth0

 

###########################################################################

这里出现了个很2的问题，折腾了我很久，一直以为是配置有错误，后来发现是正常的。
从上面查询的主的状态看很正常，然后跑到从的上去查看状态：
ip a | grep eth0
2: eth0: mtu 1500 qdisc pfifo_fast qlen 1000
    inet 10.1.1.192/24 brd 10.1.1.255 scope global eth0
奇怪了，和主的不一样，怎么修改配置文件，重启都没用。查看从的LOG
tail -f /var/log/message
Nov  7 15:34:08 nginx2 Keepalived: Starting Keepalived v1.1.17 (11/07,2012)
Nov  7 15:34:08 nginx2 Keepalived_vrrp: Using MII-BMSR NIC polling thread...
Nov  7 15:34:08 nginx2 Keepalived: Starting VRRP child process, pid=9834
Nov  7 15:34:08 nginx2 Keepalived_vrrp: Registering Kernel netlink reflector
Nov  7 15:34:08 nginx2 Keepalived_vrrp: Registering Kernel netlink command channel
Nov  7 15:34:08 nginx2 Keepalived_vrrp: Registering gratutious ARP shared channel
Nov  7 15:34:08 nginx2 Keepalived_vrrp: Opening file '/etc/keepalived/keepalived.conf'.
Nov  7 15:34:08 nginx2 Keepalived_vrrp: Configuration is using : 60997 Bytes
Nov  7 15:34:08 nginx2 Keepalived_vrrp: VRRP_Instance(VI_1) Entering BACKUP STATE
Nov  7 15:34:08 nginx2 Keepalived_vrrp: VRRP sockpool: [ifindex(2), proto(112), fd(9,10)]

再看下主的LOG
Nov  7 15:28:55 nginx1 Keepalived: Starting VRRP child process, pid=10139
Nov  7 15:28:55 nginx1 Keepalived_vrrp: Registering Kernel netlink reflector
Nov  7 15:28:55 nginx1 Keepalived_vrrp: Registering Kernel netlink command channel
Nov  7 15:28:55 nginx1 Keepalived_vrrp: Registering gratutious ARP shared channel
Nov  7 15:28:56 nginx1 Keepalived_vrrp: Opening file '/etc/keepalived/keepalived.conf'.
Nov  7 15:28:56 nginx1 Keepalived_vrrp: Configuration is using : 60999 Bytes
Nov  7 15:28:56 nginx1 Keepalived_vrrp: VRRP sockpool: [ifindex(2), proto(112), fd(9,10)]
Nov  7 15:28:57 nginx1 Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
Nov  7 15:28:58 nginx1 Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
Nov  7 15:28:58 nginx1 Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Nov  7 15:28:58 nginx1 Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.1.1.190

实在找不到原因了，就随便测试下，停了主的keepalive，额，再看下从的状态
ip a | grep eth0
2: eth0: mtu 1500 qdisc pfifo_fast qlen 1000
    inet 10.1.1.192/24 brd 10.1.1.255 scope global eth0
    inet 10.1.1.190/32 scope global eth0
能看到设置的VRRP了。因为主的keepalive停了，程序检测到有异常，就让从接管了吧。

###############################################################################

 

二、测试情况
测试1，拿一台机器ping 10.1.1.190，然后停掉主（10.1.1.191）的keepalive，在丢失1-2个包的情况下，正常切换到了从（10.1.1.192）上。
通过停keepalive前后的ssh操作可以确定分别连接到两台机器。

 

测试2，拿一台机器ping 10.1.1.190，然后断开主（10.1.1.191）的网络（/etc/init.d/network stop）,在丢失1-2个包的情况下，正常切换到了从（10.1.1.192）上。
通过断开网络前后的ssh操作可以确定分别连接到两台机器。

 

测试3，访问10.1.1.190的80服务（前提是10.1.1.191和10.1.1.192上的80服务开启），停掉10.1.1.191上的80服务（我这里是Killall nginx），再访问10.1.1.190的80服务，
无法正常访问。

网上流传着高人使用的脚本，大意就是每5秒判断一次nginx进程是否存在，如果存在就不做任何操作，如果不存在就重启nginx，重启后再判断一次nginx是否存在，如果存在
就不做操作，如果不存在kill掉keepalive，这样VRRP也会切换到从上。
vim /root/nginx_pid.sh  
#!/bin/bash 
while  : 
do
  nginxpid=`ps -C nginx --no-header | wc -l`  
  if [ $nginxpid -eq 0 ];then
    /usr/local/nginx/sbin/nginx   
    sleep 5   
    nginxpid=`ps -C nginx --no-header | wc -l`  
    if [ $nginxpid -eq 0 ];then    
    /etc/init.d/keepalived stop    
    fi
  fi
  sleep 5  
done

 

 

 

 

 

 

补充：

    重新找了下资料，nginx程序出现问题的时候对keepalive的切换可以这么做：

修改keepalive的配置文件,添加红字部分：

vim /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script check_nginx {
     script "/etc/keepalived/check_nginx.sh"
     interval 2
     weight 2
     }
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
#backup ipaddress
mcast_src_ip 10.1.1.191
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass ufsoft
}
track_script {
     check_nginx
     }
virtual_ipaddress {
10.1.1.190
}
}
在配置文件中增加的/etc/keepalived/check_nginx.sh内容如下：

#!/bin/bash
if [ "$(ps -ef | grep "nginx: master process"| grep -v grep )" == "" ]

 then
  killall -9  keepalived

fi

判断nginx进程是否存在，如果不存在则kill掉keepalived进程，这样实现了nginx程序出现异常时的keepalive切换，不用单独再跑一个脚本来做nginx验证操作了。

 

经过测试，killall掉nginx后，keepalive也会退出。另外验证80服务也能正常切换。