mongodb3.0.7扩容存储记录
扩容方案
扩容前复制集
原有db01、db02、230为复制集，其中db01为primary，230为远程复制节点。
扩容前存储
扩容方法
尽量减少线上停机时间（5分钟左右）
-
新增一台db03，采用ssm管理磁盘，初步分配1.5T磁盘空间，后续空间不足继续增加磁盘交给ssm管理和扩容
-
db03扩容完成后，上线到原有复制集中
-
db03同步数据完成后切换primary到db03
-
之后分别下线db01、db02、230等进行磁盘扩容
db03安装和配置
mongodb的安装
[root@db03 ~]# /etc/yum.repos.d/mongodb-org-3.4.repo
-bash: /etc/yum.repos.d/mongodb-org-3.4.repo: No such file or directory
[root@db03 ~]# vi /etc/yum.repos.d/mongodb-org-3.4.repo
[root@db03 ~]# yum install -y mongodb-org
Loaded plugins: fastestmirror, langpacks
mongodb-org-3.4 | 2.5 kB 00:00:00
mongodb-org-3.4/7/primary_db | 25 kB 00:00:01
Loading mirror speeds from cached hostfile
Resolving Dependencies
--> Running transaction check
---> Package mongodb-org.x86_64 0:3.4.4-1.el7 will be installed
--> Processing Dependency: mongodb-org-tools = 3.4.4 for package: mongodb-org-3.4.4-1.el7.x86_64
--> Processing Dependency: mongodb-org-shell = 3.4.4 for package: mongodb-org-3.4.4-1.el7.x86_64
--> Processing Dependency: mongodb-org-server = 3.4.4 for package: mongodb-org-3.4.4-1.el7.x86_64
--> Processing Dependency: mongodb-org-mongos = 3.4.4 for package: mongodb-org-3.4.4-1.el7.x86_64
--> Running transaction check
---> Package mongodb-org-mongos.x86_64 0:3.4.4-1.el7 will be installed
---> Package mongodb-org-server.x86_64 0:3.4.4-1.el7 will be installed
---> Package mongodb-org-shell.x86_64 0:3.4.4-1.el7 will be installed
---> Package mongodb-org-tools.x86_64 0:3.4.4-1.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
==============================================================================================
Package Arch Version Repository Size
==============================================================================================
Installing:
mongodb-org x86_64 3.4.4-1.el7 mongodb-org-3.4 5.8 k
Installing for dependencies:
mongodb-org-mongos x86_64 3.4.4-1.el7 mongodb-org-3.4 12 M
mongodb-org-server x86_64 3.4.4-1.el7 mongodb-org-3.4 20 M
mongodb-org-shell x86_64 3.4.4-1.el7 mongodb-org-3.4 11 M
mongodb-org-tools x86_64 3.4.4-1.el7 mongodb-org-3.4 49 M
Transaction Summary
==============================================================================================
Install 1 Package (+4 Dependent packages)
Total download size: 91 M
Installed size: 257 M
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
mongodb-org-server-3.4.4-1.el7 FAILED 13:33:40 ETA
[Errno 12] Timeout on (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds')
Trying other mirror.
mongodb-org-mongos-3.4.4-1.el7 FAILED 00:29:43 ETA
[Errno 12] Timeout on (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds')
Trying other mirror.
warning: /var/cache/yum/x86_64/7/mongodb-org-3.4/packages/mongodb-org-shell-3.4.4-1.el7.x86_64.rpm: Header V3 RSA/SHA1 Signature, key ID a15703c6: NOKEY
Public key for mongodb-org-shell-3.4.4-1.el7.x86_64.rpm is not installed
(1/4): mongodb-org-shell-3.4.4-1.el7.x86_64.rpm | 11 MB 00:03:23
(2/4): mongodb-org-tools-3.4.4-1.el7.x86_64.rpm | 49 MB 00:26:42
(3/4): mongodb-org-server-3.4.4-1.el7.x86_64.rpm | 20 MB 00:00:01
(4/4): mongodb-org-mongos-3.4.4-1.el7.x86_64.rpm | 12 MB 00:00:52
----------------------------------------------------------------------------------------------
Total 42 kB/s | 91 MB 00:37:17
Retrieving key from
Importing GPG key 0xA15703C6:
Userid : "MongoDB 3.4 Release Signing Key "
Fingerprint: 0c49 f373 0359 a145 1858 5931 bc71 1f9b a157 03c6
From :
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : mongodb-org-tools-3.4.4-1.el7.x86_64 1/5
Installing : mongodb-org-mongos-3.4.4-1.el7.x86_64 2/5
Installing : mongodb-org-shell-3.4.4-1.el7.x86_64 3/5
Installing : mongodb-org-server-3.4.4-1.el7.x86_64 4/5
Created symlink from /etc/systemd/system/multi-user.target.wants/mongod.service to /usr/lib/systemd/system/mongod.service.
Installing : mongodb-org-3.4.4-1.el7.x86_64 5/5
Verifying : mongodb-org-server-3.4.4-1.el7.x86_64 1/5
Verifying : mongodb-org-3.4.4-1.el7.x86_64 2/5
Verifying : mongodb-org-shell-3.4.4-1.el7.x86_64 3/5
Verifying : mongodb-org-mongos-3.4.4-1.el7.x86_64 4/5
Verifying : mongodb-org-tools-3.4.4-1.el7.x86_64 5/5
Installed:
mongodb-org.x86_64 0:3.4.4-1.el7
Dependency Installed:
mongodb-org-mongos.x86_64 0:3.4.4-1.el7 mongodb-org-server.x86_64 0:3.4.4-1.el7
mongodb-org-shell.x86_64 0:3.4.4-1.el7 mongodb-org-tools.x86_64 0:3.4.4-1.el7
Complete!
创建mongodb存储
查看存储
[root@db03 ~]# ssm list
----------------------------------
Device Total Mount point
----------------------------------
/dev/vda 20.00 GB PARTITIONED
/dev/vda1 20.00 GB /
/dev/vdb 500.00 GB
/dev/vdc 1000.00 GB
----------------------------------
---------------------------------------------------------------------
Volume Volume size FS FS size Free TypeMount point
---------------------------------------------------------------------
/dev/vda1 20.00 GB ext3 20.00 GB 17.19 GB part/
/dev/vdc 1000.00 GB xfs 999.51 GB 999.41 GB
---------------------------------------------------------------------
建立ssm管理的存储池centos
[root@db03 ~]# ssm add -p centos /dev/vdb
Physical volume "/dev/vdb" successfully created.
Volume group "centos" successfully created
[root@db03 ~]# ssm add -p centos /dev/vdc
Filesystem 'xfs' detected on the device '/dev/vdc'! Use anyway (N/y/q) ? Y
Physical volume "/dev/vdc" successfully created.
Volume group "centos" successfully extended
检查确认存储池为1.46T
[root@db03 ~]# ssm list
---------------------------------------------------------------
Device Free Used Total Pool Mount point
---------------------------------------------------------------
/dev/vda 20.00 GB PARTITIONED
/dev/vda1 20.00 GB /
/dev/vdb 500.00 GB 0.00 KB 500.00 GB centos
/dev/vdc 1000.00 GB 0.00 KB 1000.00 GB centos
---------------------------------------------------------------
------------------------------------------------
Pool Type Devices Free Used Total
------------------------------------------------
centos lvm 2 1.46 TB 0.00 KB 1.46 TB
------------------------------------------------
-------------------------------------------------------------------
Volume Volume size FS FS size Free TypeMount point
-------------------------------------------------------------------
/dev/vda1 20.00 GB ext3 20.00 GB 17.19 GB part/
-------------------------------------------------------------------
创建挂载点
[root@db03 ~]# ssm create -n goluk -s 1.46T --fstype xfs -p centos /goluk
Rounding up size to full physical extent 1.46 TiB
Logical volume "goluk" created.
meta-data=/dev/centos/goluk isize=512 agcount=4, agsize=97979136 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=0, sparse=0
data = bsize=4096 blocks=391916544, imaxpct=5
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=191365, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
确认挂载成功
[root@db03 ~]# ssm list
---------------------------------------------------------------
Device Free Used Total Pool Mount point
---------------------------------------------------------------
/dev/vda 20.00 GB PARTITIONED
/dev/vda1 20.00 GB /
/dev/vdb 4.95 GB 495.05 GB 500.00 GB centos
/dev/vdc 0.00 KB 1000.00 GB 1000.00 GB centos
---------------------------------------------------------------
------------------------------------------------
Pool Type Devices Free Used Total
------------------------------------------------
centos lvm 2 4.95 GB 1.46 TB 1.46 TB
------------------------------------------------
-------------------------------------------------------------------------------------
Volume Pool Volume size FS FS size Free Type Mount point
-------------------------------------------------------------------------------------
/dev/centos/goluk centos 1.46 TB xfs 1.46 TB 1.46 TB linear /goluk
/dev/vda1 20.00 GB ext3 20.00 GB 17.19 GB part /
-------------------------------------------------------------------------------------
修改mongodb配置文件
[root@db03 ~]# cd /etc/
[root@db03 etc]# mv mongod.conf mongod.conf.20170505
[root@db03 etc]# vi /etc/mongod.conf
[root@db03 etc]# cat /etc/mongod.conf
systemLog:
destination: file
path: "/goluk/mongo/data/mongod.log"
logAppend: true
storage:
journal:
enabled: true
dbPath: "/goluk/mongo/data/"
net:
port: 27017
bindIp: 10.0.1.33,127.0.0.1
processManagement:
fork: true
replication:
replSetName: goluk
创建目录和分配权限
[root@db03 etc]# mkdir -p /goluk/mongo/{data,conf}
[root@db03 etc]# chown -R mongod:mongod /goluk/mongo/
db03节点启动
mongod -f /etc/mongod.conf
primary节点操作
primary节点切换过程
切换检查列表
原环境需要考虑到的因素
受影响主机列表
需要听证的hosts文件内容
将原db01的主机记录修改为db03
需要重启的进程列表
-
应用服务列表，以当前运行中的为准
-
重启命令列表，备重启用
切换完成后的测试
-
db03切换为primary
-
应用测试是否正常
-
mongodb备份脚本是否修改
-
zabbix监控是否增加db03的监控项目
mongodb切换过程
primary从db01换到db03
primary降级（大约1分钟左右完成）
-
31 mongo shell 执行
rs.status() 确认状态
31 primary
32 secondary
33 secondary
230 rocovering
-
32 执行 rs.freeze(120)
120秒内db02不能成为primary节点
-
31 执行 rs.stepdown(120)
db01降级
-
db03 自动成为primary节点
更换hosts和dns记录
更换 web01、web02、cp01、cp02的hosts文件，过程略。
检查db03
-
rs.status()
-
查看连接数 db.serverStatus()
ops01备份脚本修改
扩容遇到问题
原节点mongodb都是3.0.7的版本，原计划db03升级到3.4，发现复制集不能连接上原复制集，降级到3.0.14版本才完成本次扩容
阅读(8177) | 评论(0) | 转发(0) |
