(原创)Cloudstack 部署 keepalived高可用HA 自动切换 drbd+nfs+heartbeat 存储高可用
Master
Keepalived
A-1 192.168.1.213 cloudstack-management
A-2 192.168.1.140 cloudstack-management
A1和A2用的keepalived做的高可用 VIP地址为192.168.1.188 把安装好的 /etc/cloudstack/ cp 到 存储上 同时挂载存储到 /etc/cloudstack/ 出现故障keepalived 自动切换并自动执行启动脚本。注 主备不能同时启动 cloudstack-management
Agent
B-1 192.168.1.121 必须配置好网卡 和kvm
DB
C-1 192.168.1.136 必须授权
GRANT ALL PRIVILEGES ON *.* TO ‘root’@’%’ IDENTIFIED BY ‘cloudstack’ WITH GRANT OPTION;
GRANT ALL PRIVILEGES ON *.* TO ‘clouck’@’%’ IDENTIFIED BY ‘cloudstack’ WITH GRANT OPTION;
存储
用的是NFS+DRBD+heartbeat架构 测试结果 当一台服务器down机 另一台接管nfs和DRBD
D-1 192.168.1.191
D-2 192.168.1.108
vip 192.168.1.109
配置 nfs
/home/drbd *(insecure,rw,no_root_squash,no_subtree_check)
mkdir /home/drbd/nfs1 /home/drbd/nfs2 这里为Cloudstack的主存储 和二级存储
重要 安装前保证 /home/drbd/nfs1 /home/drbd/nfs2 为空
高可用安装过程
首先配置存储
安装nfs
sudo apt-get install nfs-kernel-server
配置 /etc/exports
/home/drbd *(insecure,rw,no_root_squash,no_subtree_check)
安装drbd
apt-get install -y –force-yes drbd8
配置
/etc/drbd.d/r0.res
resource nft {
device /dev/drbd0;
disk “/dev/sdc”;
meta-disk internal;
on drbd1 {
address ipv4 192.168.1.191:7700;
}
on drbd2 {
address ipv4 192.168.1.108:7700;
}
}
cp到备机上
drbdadm create-md nft
drbdadm — –force primary nfs
查看 /etc/init.d/drbd status
m:res cs ro ds p mounted fstype
0:nft Connected Primary/Secondary UpToDate/UpToDate C /home/drbd ext4
注 Primary则为主 Secondary 则为从
重要
在主DRBD设备中可以进行不受限制的读和写的操作。他可用来创建和挂载文件系统、初始化或者是直接I/O的快设备，等等。
在次DRBD设备中，接收所有来自对等节点的更新，但是与此同时也就完全拒绝了访问。它即不能被应用也不能被读写访问。为次节不能被读写访问是为了保持缓冲一致性，这就意味着次节点是不可能以任何形式被访问的。
最后配置heartbeat
/etc/ha.d
cat authkeys
auth 3
3 md5 9bf2f23aae3a63c16ea681295ffd7666
cat ha.cf 脚本太长 请自行度娘
cat haresources
drbd1 IPaddr::192.168.1.109/24/eth0:0 drbddisk::nft Filesystem::/dev/drbd1::/home/drbd::ext4 killnfsd
现需要二个启动文件在 resource.d里
cat killnfsd
killall -9 nfsd; /etc/init.d/nfs-kernel-server restart; exit 0
cat drbddisk 脚本太长 请自行度娘
cp到备机上
/etc/init.d/nfs-kernel-server start
/etc/init.d/drbd start
/etc/init.d/heartbeat start
安装cloudstack-management
echo “deb precise 4.5″ >> /etc/apt/sources.list.d/cloudstack.list
#配置CloudStack官方源证书
wget -O – |apt-key add –
apt-get update
apt-get install cloudstack-management -y
全部依赖会自动安装
安装keepalived
apt-get install libssl-dev
apt-get install openssl
apt-get install libpopt-dev
在主机和备用机安装keepalived，命令：
apt-get install keepalived
vi /etc/keepalived/keepalived.conf 脚本太长 请到环境里查看
cat cloudstack_master.sh
mount 192.168.1.109:/home/drbd/manager /etc/cloudstack/
/etc/init.d/cloudstack-management start
cat cloudstack_slave.sh
/etc/init.d/cloudstack-management stop
记得在DB授权先
cloudstack-setup-databases cloud:cloudstack@192.168.1.136 –deploy-as=root:cloudstack -e file -m cloudstack -k cloudstack -i “192.168.1.188”
192.168.1.188为VIP
二台机都执行
cloudstack-setup-management
把安装好的 /etc/cloudstack/ cp 到存储的一个目录里 同时挂载存储到 /etc/cloudstack/
配置 DB
DEBIAN_FRONTEND=’noninteractive’ apt-get -o Dpkg::Options::=’–force-confold’ install mysql-server -y
#在[mysqld]模块中加入以下参数 具体官方文档说明 (docs.cloudstack.apache.org/projects/cloudstack-installation/en/4.5/management-server/index.html?highlight=ubuntu#install-on-ubuntu)
sed -i ‘/Basic Settings/i\innodb_rollback_on_timeout=1′ /etc/mysql/my.cnf
sed -i ‘/Basic Settings/i\innodb_lock_wait_timeout=600′ /etc/mysql/my.cnf
sed -i ‘/Basic Settings/i\max_connections=350′ /etc/mysql/my.cnf
sed -i ‘/Basic Settings/i\log-bin=mysql-bin’ /etc/mysql/my.cnf
sed -i “/Basic Settings/i\binlog-format = ‘ROW'” /etc/mysql/my.cnf
service mysql restart
#修改DB密码
mysqladmin -u root password “cloudstack”
挂载存储 创建个cloudstack主目录 二级目录
准备System VM Template我们选择KVM作为虚拟化引擎 准备System VM Template我们选择KVM作为虚拟化引擎
/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt -m /mnt/secondary -u -h kvm -F
下载速度很慢 这个 请加到115离线 上传上去 把-u 修改成-f
授权
GRANT ALL PRIVILEGES ON *.* TO ‘root’@’%’ IDENTIFIED BY ‘cloudstack’ WITH GRANT OPTION;
GRANT ALL PRIVILEGES ON *.* TO ‘clouck’@’%’ IDENTIFIED BY ‘cloudstack’ WITH GRANT OPTION;
配置 agent
#配置CloudStack官方源
echo “deb precise 4.5″ >> /etc/apt/sources.list.d/cloudstack.list
#配置CloudStack官方源证书
wget -O – |apt-key add –
apt-get update
apt-get install qemu-kvm libvirt-bin virt-manager bridge-utils
安装配置KVM虚拟化Host主机
apt-get install cloudstack-agent
#安装配置libvirt
echo ‘listen_tls = 0
listen_tcp = 1
tcp_port = “16509”
mdns_adv = 0
unix_sock_group = “libvirtd”
unix_sock_rw_perms = “0770”
auth_unix_ro = “none”
auth_unix_rw = “none”
auth_tcp = “none” ‘ > /etc/libvirt/libvirtd.conf
#修改以下参数
sed -i ‘s/env libvirtd_opts=”-d”/env libvirtd_opts=”-d -l”/g’ /etc/init/libvirt-bin.conf
sed -i ‘s/#vnc_listen = “0.0.0.0” /vnc_listen = “0.0.0.0”/g’ /etc/libvirt/qemu.conf
/etc/init.d/libvirt-bin restart
配置网卡
参考
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet manual
auto virbr0
auto cloudbr0
iface cloudbr0 inet static
address 192.168.1.121
netmask 255.255.255.0
gateway 192.168.1.1
bridge_ports eth0
bridge_fd 5
bridge_stp off
bridge_maxwait 1
dns-nameservers 114.114.114.114
# Private network
auto cloudbr1
iface cloudbr1 inet manual
bridge_ports eth0
bridge_fd 5
bridge_stp off
bridge_maxwait 1
重启网卡
/etc/init.d/cloudstack-agent start
master启动
/etc/init.d/cloudstack-management start
完毕
转载请注明 白菜营养好
高级运维工程师 曹青峰 QQ 258896473
- 本文固定链接: 部署-keepalived高可用ha-自动切换-drbdnfheartbeat-存储高可用/
- 转载请注明: cqf 于 曹青峰的博客 发表