9200 port strange problem!!

在配置keepalived+haproxy+xtradbcluster时遇到一个问题，9200端口已经启动了，通过lsof -i:9200 有输出，但curl -I  始终提示503 错误：我在percona论坛发帖提问，后来自己解决了！我的提问如下：

Hi,
I want to establish a cluster with 5 nodes (haproxy+keepalived and 3 nodes)
I occured a problem in health check for the cluster
I have configured as follows in three nodes:
[root@localhost init.d]# cd /etc/xinetd.d/
[root@localhost xinetd.d]# pwd
/etc/xinetd.d
[root@localhost xinetd.d]# ll mysqlchk
-rw-r--r-- 1 root root 1090 Nov 14 17:49 mysqlchk
[root@localhost xinetd.d]# ll /etc/xinetd.d/mysqlchk
-rw-r--r-- 1 root root 1090 Nov 14 17:49 /etc/xinetd.d/mysqlchk
[root@localhost xinetd.d]# cat mysqlchk
# default: on
# description: mysqlchk
service mysqlchk
{
# this is a config for xinetd, place it in /etc/xinetd.d/
disable = no
flags = REUSE
socket_type = stream
port = 9200
wait = no
user = nobody
server = /usr/bin/clustercheck
server_args = sstuser s3cret 1 /var/log/clusterlog 1 /etc/my.cnf
log_on_failure += USERID
only_from = 0.0.0.0/0
#
# Passing arguments to clustercheck
# <user> <pass> <available_when_donor=0|1> <log_file> <available_when_readonly=0|1> <defaults_extra_file>"
# Recommended: server_args = user pass 1 /var/log/log-file 0 /etc/my.cnf.local"
# Compatibility: server_args = user pass 1 /var/log/log-file 1 /etc/my.cnf.local"
# 55-to-56 upgrade: server_args = user pass 1 /var/log/log-file 0 /etc/my.cnf.extra"
#
# recommended to put the IPs that need
# to connect exclusively (security purposes)
per_source = UNLIMITED
}
[root@localhost xinetd.d]# lsof -i:9200
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
xinetd 2446 root 5u IPv4 7047 0t0 TCP *:mysqlchk (LISTEN)
[root@localhost xinetd.d]# /usr/bin/clustercheck
HTTP/1.1 200 OK
Content-Type: text/plain
Connection: close
Content-Length: 42

Percona XtraDB Cluster Node is synced.

[root@localhost xinetd.d]# curl -I localhost:9200
HTTP/1.1 503 Service Unavailable
Content-Type: text/plain
Connection: close
Content-Length: 44

[root@localhost xinetd.d]# curl localhost:9200 ##here It does not
Percona XtraDB Cluster Node is not synced.


[root@localhost xinetd.d]# iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
[root@localhost xinetd.d]# getenforce
Disabled


I get 200 response with /usr/bin/clustercheck
But I get 503 response with curl localhost:9200

Could you tell me why ?
many

I am sure ,the three nodes are synced as follows:
node1:
mysql> show status like 'wsrep%';
.....
| wsrep_local_state_comment | Synced
......
node2:
mysql> show status like 'wsrep%';
.....
| wsrep_local_state_comment | Synced
......

node3:
mysql> show status like 'wsrep%';
.....
| wsrep_local_state_comment | Synced
......

I have fix the problem. I modified the script clustercheck.
I replace mysql with /usr/local/mysql/bin/mysql in /usr/bin/clustercheck script.
then It works

as follows:
if [[ -r $DEFAULTS_EXTRA_FILE ]];then
MYSQL_CMDLINE="/usr/local/mysql/bin/mysql --defaults-extra-file=$DEFAULTS_EXTRA_FILE -nNE --connect-timeout=$TIMEOUT \
${EXTRA_ARGS}"
else
MYSQL_CMDLINE="/usr/local/mysql/bin/mysql -nNE --connect-timeout=$TIMEOUT ${EXTRA_ARGS}"
fi
although it  I am confused about it .Because I have add mysql bin into the PATH
as follows:
[root@db132 bin]# echo $PATH
/usr/local/xtrabackup/bin:/usr/local/mysql/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
[root@db132 bin]#


Could anyone tell me why？
many

问题虽然解决了，但还是有点不明白，但无论怎么说，问题总是解决了！
处理问题思路如下：
打开haproxy的web界面会发现，所有的机器都是down的状态，
这是因为percona xtradbcluster通过9200端口来检测集群的工作状态，还需要配置9200 端口，
配置过程如下：
yum -y install xinetd
cp /usr/local/mysql/bin/clustercheck    /usr/bin
cp  /usr/local/mysql/xinetd.d/mysqlchk  /etc/xinnetd.d
vi /etc/xinnetd.d/mysqlchk
在
 server          = /usr/bin/clustercheck
行下面添加如下：
server_args = sstuser s3cret 1 /var/log/clusterlog 1 /etc/my.cnf   ##sstuser我已经添加了
然后重启xinetd服务，查看9200端口是否已经启动
将mysqlchk        9200/tcp                # mysqlchk行，添加到/etc/services文件中


在这里遇到一个奇怪的问题，
我能将9200端口给启动起来，通过lsof -i:9200 输出是正常的 
但是curl -I 却提示503 ， 在网上也没有找到解决方法，后来我仔细查看了clustercheck脚本，
发现curl -I 直接执行的是
else 
    # Percona XtraDB Cluster node local state is not 'Synced' => return HTTP 503
    # Shell return-code is 1
    echo -en "HTTP/1.1 503 Service Unavailable\r\n" 
    echo -en "Content-Type: text/plain\r\n" 
    echo -en "Connection: close\r\n" 
    echo -en "Content-Length: 44\r\n" 
    echo -en "\r\n" 
    echo -en "Percona XtraDB Cluster Node is not synced.\r\n" 
    sleep 0.1
    exit 1
fi 
根本没有执行clusterworker中的if上半部分，但是我用mysql 命令执行,在机器上执行的时候是执行的if上半部分
而且
[root@db131 ~]# /usr/bin/clustercheck
HTTP/1.1 200 OK
Content-Type: text/plain
Connection: close
Content-Length: 42


Percona XtraDB Cluster Node is synced.
 我怀疑是mysql的路径问题，mysql 有两个，一个是我安装的mysql，另外一个是系统自带的mysql，
其实我已经配置PATH了，而且将我配置的mysql放到PATH的前面了，理论上将应该没有问题，
[root@db143 tmp]# which -a mysql
/usr/local/mysql/bin/mysql
/usr/bin/mysql
[root@db143 tmp
这个问题困扰了我大概一上午的时间，我将mysql修改为/usr/local/mysql/bin/mysql 就正常了
然后下面就一切就顺水顺风了！