redis cluster搭建

从redis官网看最新稳定版本已经更新到3.2.6了，我这里补充一篇以前3.2.0的集群配置。
说明：

redis从3.0版本以后开始支持cluster模式，原理和功能大家可以上网查，对于运维人员来说最重要的去掉了代理层、避免出现单点故障，但是redis cluster至少需要三个主节点和三个从节点。每一对主从形成一组关系。因此需要三台服务器，每台服务器上两个实例。因为一个主对应一个从，所以一台服务器上不能同时是一个主和从，必须三台服务器分开主从，来保证服务的正常运行。我这里为了演示方便就在一台服务器上部署。注意：生产环境必须是三台机器，不然达不到高可用。

部署环境：

主机名	IP地址	操作系统版本	用途
test01	192.168.2.9	CentOS6.3（64位）	实例 7000/7001
test02	192.168.2.10	CentOS 6.3（64位）	实例 7002/7003
test03	192.168.2.7	CentOS 6.3（64位）	实例 7004/7005

部署步骤：

1、编译安装redis。

把安装包放在/data/下解压并编译

[root@test01 data]# tar -zxf redis-3.2.0.tar.gz
[root@test01 redis]# cd redis-3.2.0
[root@test01 redis-3.2.0]# make && make install
[root@test01 redis]# ln -s /data/redis/redis-3.2.0 /usr/local/redis

2、创建redis集群节点

[root@test01 local]# mkdir redis_cluster
[root@test01 local]# cd redis_cluster/
[root@test01 redis_cluster]# mkdir 7000 7001

3、复制默认的节点配置文件到集群节点中

[root@test01 redis_cluster]# cp /usr/local/redis/redis.conf ./7000

4、修改默认配置文件

[root@test01 7000]# vim redis.conf
daemonize    yes                  //redis后台运行
pidfile  /var/run/redis_7000.pid   //pidfile文件对应7000
port  7000                         //端口7000
cluster-enabled  yes              //开启集群  把注释#去掉
cluster-config-file  nodes.conf    //集群的配置  配置文件首次启动自动生成
cluster-node-timeout  6000      //请求超时
appendonly  yes                  //日志开启，环境测试开启，正式环境建议关闭
bind 192.168.2.9             //修改监听的地址为本机地址

5、修改完后把配置文件复制到7001上并修改相应的端口。

[root@test01 7000]# cp redis.conf	../7001/

6、另外两台服务器重复上面配置。

7、启动各个节点，注意一定要到端口目录下去启动，不然有的启动不起来。

[root@test01 7000]# /usr/local/redis/src/redis-server redis.conf
[root@test01 7000]# cd ../7001/
[root@test01 7001]# /usr/local/redis/src/redis-server redis.conf

查看启动进程

[root@test01 7001]# ps -ef|grep redis
root      8858     1  0 11:38 ?        00:00:00 /usr/local/redis/src/redis-server 192.168.2.9:7000 [cluster]
root      8865     1  0 11:39 ?        00:00:00 /usr/local/redis/src/redis-server 192.168.2.9:7001 [cluster]
root      8870 27799  0 11:39 pts/1    00:00:00 grep redis

再分别启动其余服务器节点。

8、创建集群必须的软件，三台服务器都执行

[root@test01 7000]# yum -y install ruby ruby-devel rubygems rpm-build
[root@test01 7000]# ruby -v
ruby 1.8.7 (2013-06-27 patchlevel 374) [x86_64-linux]
[root@redis 7000]# rpm -qa|grep rubyge
rubygems-1.3.7-5.el6.noarch
[root@redis 7000]# gem install redis
Successfully installed redis-3.3.0
1 gem installed
Installing ri documentation for redis-3.3.0...
Installing RDoc documentation for redis-3.3.0.

9、确认所有的节点都启动，接下来使用参数create 创建

[root@test01 7000]# /usr/local/redis/src/redis-trib.rb create --replicas 1 192.168.2.9:7000 192.168.2.10:7002 192.168.2.7:7004 192.168.2.10:7003 192.168.2.7:7005 192.168.2.9:7001
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.2.7:7004
192.168.2.9:7000
192.168.2.10:7002
Adding replica 192.168.2.9:7001 to 192.168.2.7:7004
Adding replica 192.168.2.7:7005 to 192.168.2.9:7000
Adding replica 192.168.2.10:7003 to 192.168.2.10:7002
M: bede5b72dfbb5274e52a1f0c5f6b43170afec8af 192.168.2.9:7000
slots:5461-10922 (5462 slots) master
M: 01594f84df9e743a74a47f9aaa58fa41402dfe25 192.168.2.10:7002
slots:10923-16383 (5461 slots) master
M: 27cedfdc0a648b9141736f156a4d89828d7bf695 192.168.2.7:7004
slots:0-5460 (5461 slots) master
S: 6f13ca12a9be3b0c093d02c81fed337307f295af 192.168.2.10:7003
replicates 01594f84df9e743a74a47f9aaa58fa41402dfe25
S: 00333fa0ac74863e86c3108f6040abe1183a2b9b 192.168.2.7:7005
replicates bede5b72dfbb5274e52a1f0c5f6b43170afec8af
S: f167b98d8f78bfdb4c1823c0d6be7f1a12aff194 192.168.2.9:7001
replicates 27cedfdc0a648b9141736f156a4d89828d7bf695
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 192.168.2.9:7000)
M: bede5b72dfbb5274e52a1f0c5f6b43170afec8af 192.168.2.9:7000
slots:5461-10922 (5462 slots) master
M: 01594f84df9e743a74a47f9aaa58fa41402dfe25 192.168.2.10:7002
slots:10923-16383 (5461 slots) master
M: 27cedfdc0a648b9141736f156a4d89828d7bf695 192.168.2.7:7004
slots:0-5460 (5461 slots) master
M: 6f13ca12a9be3b0c093d02c81fed337307f295af 192.168.2.10:7003
slots: (0 slots) master
replicates 01594f84df9e743a74a47f9aaa58fa41402dfe25
M: 00333fa0ac74863e86c3108f6040abe1183a2b9b 192.168.2.7:7005
slots: (0 slots) master
replicates bede5b72dfbb5274e52a1f0c5f6b43170afec8af
M: f167b98d8f78bfdb4c1823c0d6be7f1a12aff194 192.168.2.9:7001
slots: (0 slots) master
replicates 27cedfdc0a648b9141736f156a4d89828d7bf695
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

--replicas 1  创建集群指定一个从节点。

因为创建集群的时候不能指定主从，它是随机的。加入集群的节点顺序不分前后，如果一台服务器上正好有主节点和从节点，而且是一组的话，必须将同一台服务器上的主从节点踢掉，换成不同服务器的节点。
从上面可以看到很不辛7003和7002分配了一台服务器上。经过测试一般都会有一台服务器上存在同一组的主和从节点。
主从关系图：
192.168.2.10:7002        192.168.2.10:7003
192.168.2.9:7000         192.168.2.7:7005
192.168.2.7:7004         192.168.2.9:7001

10、我们把192.168.2.10:7003和192.168.2.7:7005调换下。

删除7003节点

[root@test01 7000]# /usr/local/redis/src/redis-trib.rb del-node 192.168.2.10:7003 6f13ca12a9be3b0c093d02c81fed337307f295af
>>> Removing node 6f13ca12a9be3b0c093d02c81fed337307f295af from cluster 192.168.2.10:7003
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

del-node  删除节点后面跟主机和对应的端口还有实例的id。这样才能删除。

11、修改192.168.2.7:7005的主为192.168.2.10:7002

进入192.168.2.7:7005的redis

[root@test02 redis_cluster]# /usr/local/redis/src/redis-cli -c -p 7005 -h 192.168.2.7
192.168.2.7:7005> cluster replicate 01594f84df9e743a74a47f9aaa58fa41402dfe25 #新主的id
OK

查看现在主从状态

[root@test02 redis_cluster]# /usr/local/redis/src/redis-trib.rb check 192.168.2.10:7002
>>> Performing Cluster Check (using node 192.168.2.10:7002)
M: 01594f84df9e743a74a47f9aaa58fa41402dfe25 192.168.2.10:7002
slots:10923-16383 (5461 slots) master
1 additional replica(s)
M: bede5b72dfbb5274e52a1f0c5f6b43170afec8af 192.168.2.9:7000
slots:5461-10922 (5462 slots) master
0 additional replica(s)
M: 27cedfdc0a648b9141736f156a4d89828d7bf695 192.168.2.7:7004
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: f167b98d8f78bfdb4c1823c0d6be7f1a12aff194 192.168.2.9:7001
slots: (0 slots) slave
replicates 27cedfdc0a648b9141736f156a4d89828d7bf695
S: 00333fa0ac74863e86c3108f6040abe1183a2b9b 192.168.2.7:7005
slots: (0 slots) slave
replicates 01594f84df9e743a74a47f9aaa58fa41402dfe25
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

可以看到主从关系已经改变。

12、再添加刚删除的7003节点。

[root@test02 redis_cluster]# ps -ef|grep redis
root      4441     1  0 11:45 ?        00:00:13 /usr/local/redis/src/redis-server 192.168.2.10:7002 [cluster]
root      4673  4275  0 15:02 pts/0    00:00:00 grep redis
[root@test02 redis_cluster]# cd 7003/
[root@test02 7003]# ls
appendonly.aof  dump.rdb  nodes.conf  redis.conf
[root@test02 7003]# rm -f appendonly.aof dump.rdb nodes.conf
[root@test02 7003]# /usr/local/redis/src/redis-server redis.conf
[root@test02 7003]# ps -ef|grep redis
root      4441     1  0 11:45 ?        00:00:14 /usr/local/redis/src/redis-server 192.168.2.10:7002 [cluster]
root      4677     1  0 15:02 ?        00:00:00 /usr/local/redis/src/redis-server 192.168.2.10:7003 [cluster]
root      4681  4275  0 15:02 pts/0    00:00:00 grep redis

13、执行下列命令添加从指定的主节点。

[root@test02 7003]# /usr/local/redis/src/redis-trib.rb add-node --slave --master-id bede5b72dfbb5274e52a1f0c5f6b43170afec8af 192.168.2.10:7003 192.168.2.9:7000
>>> Adding node 192.168.2.7:7005 to cluster 192.168.2.9:7000
>>> Performing Cluster Check (using node 192.168.2.9:7000)
M: bede5b72dfbb5274e52a1f0c5f6b43170afec8af 192.168.2.9:7000
slots:5461-10922 (5462 slots) master
0 additional replica(s)
S: f167b98d8f78bfdb4c1823c0d6be7f1a12aff194 192.168.2.9:7001
slots: (0 slots) slave
replicates 27cedfdc0a648b9141736f156a4d89828d7bf695
S: 00333fa0ac74863e86c3108f6040abe1183a2b9b 192.168.2.7:7005
slots: (0 slots) slave
replicates 01594f84df9e743a74a47f9aaa58fa41402dfe25
M: 27cedfdc0a648b9141736f156a4d89828d7bf695 192.168.2.7:7004
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 01594f84df9e743a74a47f9aaa58fa41402dfe25 192.168.2.10:7002
slots:10923-16383 (5461 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.2.10:7003 to make it join the cluster.
Waiting for the cluster to join.
>>> Configure node as replica of 192.168.2.9:7000.
[OK] New node added correctly.

从上面显示7003节点成为7000的从节点。

14、查看主从现在情况

[root@test02 7003]# /usr/local/redis/src/redis-trib.rb check 192.168.2.10:7002
>>> Performing Cluster Check (using node 192.168.2.10:7002)
M: 01594f84df9e743a74a47f9aaa58fa41402dfe25 192.168.2.10:7002
slots:10923-16383 (5461 slots) master
1 additional replica(s)
M: bede5b72dfbb5274e52a1f0c5f6b43170afec8af 192.168.2.9:7000
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: 2e26f92f8dca2dfb13f159a59b6260f106bc8cb4 192.168.2.10:7003
slots: (0 slots) slave
replicates bede5b72dfbb5274e52a1f0c5f6b43170afec8af
M: 27cedfdc0a648b9141736f156a4d89828d7bf695 192.168.2.7:7004
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: f167b98d8f78bfdb4c1823c0d6be7f1a12aff194 192.168.2.9:7001
slots: (0 slots) slave
replicates 27cedfdc0a648b9141736f156a4d89828d7bf695
S: 00333fa0ac74863e86c3108f6040abe1183a2b9b 192.168.2.7:7005
slots: (0 slots) slave
replicates 01594f84df9e743a74a47f9aaa58fa41402dfe25
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered

这样就是一个正常的集群了~~运维人员再也不会因为宕机或者redis挂了的问题头疼了！