kafka集群环境安装

三台操作系统搭建kafka集群环境

OS版本：
[root@Master kafka_2.10-0.8.2.2]# lsb_release -a
LSB Version: :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Distributor ID: CentOS
Description: CentOS release 6.5 (Final)
Release: 6.5
Codename: Final

[root@Master kafka_2.10-0.8.2.2]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.56.121 Master
192.168.56.122 Slave1
192.168.56.123 Slave2

软件目录安装在：
/usr/local/kafka/kafka_2.10-0.8.2.2

Master环境的zookeeper.properties修改：

点击(此处)折叠或打开

tickTime=2000
dataDir=/data/zookeeper
clientPort=2181
maxClientCnxns=0
initLimit=5
syncLimit=2
server.1=192.168.56.121:2888:3888
server.2=192.168.56.122:2888:3888
server.3=192.168.56.123:2888:3888

Master环境的server.properties文件修改如下所示。
Slave1和Slave2环境的broker.id为2和3，host.name分别为192.168.56.122和192.168.56.123

broker.id=1
port=9092
host.name=192.168.56.121
zookeeper.connect=192.168.56.121:2181,192.168.56.122:2181,192.168.56.123:2181
log.dirs=/data/kafka-logs
producer.type=async
broker.list=192.168.56.121:9092,192.168.56.122:9092,192.168.56.123:9092

启动每台服务器的kafka：

>bin/kafka-server-start.sh config/server.properties &

测试集群

创建一个topic

>bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

查看创建的topic

[root@Slave1 kafka_2.10-0.8.2.2]# bin/kafka-topics.sh --list --zookeeper 192.168.56.121:2181
test
[root@Slave1 kafka_2.10-0.8.2.2]# bin/kafka-topics.sh --list --zookeeper 192.168.56.122:2181
test
[root@Slave1 kafka_2.10-0.8.2.2]# bin/kafka-topics.sh --list --zookeeper 192.168.56.123:2181
test

查看集群情况：

[root@Master kafka_2.10-0.8.2.2]# bin/kafka-topics.sh --describe --zookeeper 192.168.56.121:2181 --topic test Topic:test
PartitionCount:1 ReplicationFactor:1 Configs: Topic: test Partition: 0 Leader: 3 Replicas: 3 Isr: 3
[root@Master kafka_2.10-0.8.2.2]# bin/kafka-topics.sh --describe --zookeeper 192.168.56.122:2181 --topic test Topic:test
PartitionCount:1 ReplicationFactor:1 Configs: Topic: test Partition: 0 Leader: 3 Replicas: 3 Isr: 3
[root@Master kafka_2.10-0.8.2.2]# bin/kafka-topics.sh --describe --zookeeper 192.168.56.123:2181 --topic test Topic:test
PartitionCount:1 ReplicationFactor:1 Configs: Topic: test Partition: 0 Leader: 3 Replicas: 3 Isr: 3