LINUX下多路径的配置与LVM

LINUX下多路径的配置

 DM_Multipath.pdf    

LVM设置 

 Cluster_Logical_Volume_Manager.pdf   

 

多路径软件包的安装
Rpm –ivh device-mapper-1.02.13-6.9.i586.rpm
Rpm –ivh multipath-tools-0.4.7-34.18.i586.rpm

编辑文件/etc/multipath.conf
对于DS8000系统存储，可以在IBM网站上找到相关的配置文件。

启动相关模块
可以使用lsmod |grep dm_multipath来检查安装是否正常。
modprobe dm-multipath
modprobe dm-round-robin

启动服务
service multipathd start
chkconfig multipathd on

设备的设备名会被生成，/dev/dm-#，#指代的是多路径组，如果/dev/sda是唯一的多路径设备，/dev/dm-0将会是/dev/sda和/dev/sdb的多路径设备

display the multipath debug info "multipath -v3"

multipath -ll
如果mpath和dm排序不对，可先关掉multipathd服务,然后 multipath -F(flush all unused multipath device maps)，(应该就可以了？！）
然后到/var/lib/multipath找到bindings文件删除，然后重启服务。

 

创建LVM
~~~~~~~~~~~~~~~~~~~~~~~~~~
创建PV,可一条命令创建多个 PV
pvcreate /dev/dm-2 /dev/dm-3 /dev/dm-4 /dev/dm-5 /dev/dm-6 /dev/dm-7 /dev/dm-8 /dev/dm-9 /dev/dm-10 /dev/dm-11 /dev/dm-12 /dev/dm-13 /dev/dm-14 /dev/dm-15 /dev/dm-16 /dev/dm-17 /dev/dm-18 /dev/dm-19 /dev/dm-20 /dev/dm-21 /dev/dm-22 /dev/dm-23 /dev/dm-24 /dev/dm-25 /dev/dm-26 /dev/dm-27 /dev/dm-28 /dev/dm-29 /dev/dm-30 /dev/dm-31 /dev/dm-32 /dev/dm-33 /dev/dm-34 /dev/dm-35 /dev/dm-36 /dev/dm-37 /dev/dm-38 /dev/dm-39 /dev/dm-40 /dev/dm-41 /dev/dm-42

创建VG
vgcreate dataVG /dev/dm-2 /dev/dm-3 /dev/dm-4 /dev/dm-5 /dev/dm-6 /dev/dm-7 /dev/dm-8 /dev/dm-9 /dev/dm-10 /dev/dm-11 /dev/dm-12 /dev/dm-13 /dev/dm-14 /dev/dm-15 /dev/dm-16 /dev/dm-17 /dev/dm-18 /dev/dm-19 /dev/dm-20 /dev/dm-21 /dev/dm-22 /dev/dm-23 /dev/dm-24 /dev/dm-25 /dev/dm-26 /dev/dm-27 /dev/dm-28 /dev/dm-29 /dev/dm-30 /dev/dm-31 /dev/dm-32 /dev/dm-33 /dev/dm-34 /dev/dm-35 /dev/dm-36 /dev/dm-37 /dev/dm-38 /dev/dm-39 /dev/dm-40 /dev/dm-41 /dev/dm-42
删除VG
vgremoe vgname
添加PV到VG中
vgextend dataVG /dev/dm-5 /dev/dm-6 /dev/dm-7 /dev/dm-8 /dev/dm-9 /dev/dm-10 /dev/dm-11 /dev/dm-12 /dev/dm-13 /dev/dm-14

创建LV，指定PV
lvcreate -L 110g -n test dataVG /dev/mapper/mpath1 /dev/mapper/mpath2
创建 MIRROR LV
#####################其他部门  200G
lvcreate -L 102392m -m1 -n lv_otherDep dataVG /dev/mapper/mpath1  /dev/mapper/mpath17 /dev/mapper/mpath15  --nosync  
lvextend -L+102392m /dev/dataVG/lv_otherDep  /dev/mapper/mpath6  /dev/mapper/mpath22
#####################二部网站  100G
lvcreate -L 102392m -m1 -n lv_web2 dataVG /dev/mapper/mpath5  /dev/mapper/mpath21 /dev/mapper/mpath15  --nosync
mkfs.ext3 /dev/dataVG/lv_web2
#####################   mail & web 900g

lvcreate -L 102392m -m1 -n lv_mailweb dataVG /dev/mapper/mpath0  /dev/mapper/mpath16  /dev/mapper/mpath15 --nosync
lvchange -an /dev/dataVG/lv_mailweb
lvextend -L+511960m /dev/dataVG/lv_mailweb  /dev/mapper/mpath31  /dev/mapper/mpath36  /dev/mapper/mpath32  /dev/mapper/mpath37  /dev/mapper/mpath33  /dev/mapper/mpath38  /dev/mapper/mpath34  /dev/mapper/mpath39  /dev/mapper/mpath35  /dev/mapper/mpath40
lvchange -ay /dev/dataVG/lv-mailweb
lvextend -L+307176m /dev/dataVG/lv_mailweb  /dev/mapper/mpath7  /dev/mapper/mpath23  /dev/mapper/mpath8  /dev/mapper/mpath24  /dev/mapper/mpath9  /dev/mapper/mpath25

格式化文件系统
mkfs.ext3 /dev/dataVG/lv-mailweb
#####################电子商务处文档 300g
lvcreate -L 102392m -m1 -n lv_document dataVG /dev/mapper/mpath2  /dev/mapper/mpath18  /dev/mapper/mpath15 --nosync
lvchange -an /dev/dataVG/lv_document
lvextend -L+204784m /dev/dataVG/lv_document  /dev/mapper/mpath3  /dev/mapper/mpath19  /dev/mapper/mpath4  /dev/mapper/mpath20
lvchange -ay /dev/dataVG/lv_document

格式化文件系统
mkfs.ext3 /dev/dataVG/lv_document
##########################################
创建 MIRROR LV
#####################管理科服务器镜像  100G
lvcreate -L 102392m -m1 -n lv_images dataVG /dev/mapper/mpath10  /dev/mapper/mpath26 /dev/mapper/mpath15  --nosync  
###########################################################################

mount /dev/dataVG/lv_fwb /mnt/fwb/
mount /dev/dataVG/lv_web /mnt/web/
mount /dev/dataVG/lv_doc /mnt/doc/
mount /dev/dataVG/lv_mail /mnt/mail/
mount /dev/dataVG/lv_oabackup /mnt/dbbackup/

注：一定要三个以上PV，因为最后一个要放LOG
增加镜像LV，同样要三个以上PV
lvconvert -m1 /dev/dataVG/testLV /dev/mapper/mpath3 /dev/mapper/mpath4 /dev/mapper/mpath5

--m --nosync --mirrorlog
nosync,对镜像前的数据不进行同步。对于新的设备做镜像可采取此方法，快捷。
mirrorlog,

不激活某个LV
lvchange -an /dev/dataVG/test

配置reiserfs
要编译内核
先从CENTOS网站下载 kernel-2.6.18-92.el5.src.rpm
rpm -ivh kernel-2.6.18-92.el5.src.rpm
cd /usr/src/redhat/SPECS/
yum install unifdef
rpmbuild -bp kernel-2.6.spec
cd /usr/src/redhat/BUILD/kernel-2.6.18/linux-2.6.18.i386/
make mrproper
make menuconfig 找到FILE SYSTEM，找到REISERFS
yum install ncurses-devel（如果上面命令报错的话）   

make dep
make clean
make bzImage
make modules
make modules_install
启用新内核
cp /usr/src/redhat/BUILD/kernel-2.6.18/linux-2.6.18.i386/arch/i386/boot/bzImage /boot/vmlinuz-2.6.18
mkinitrd /boot/initrd-2.6.18.img 2.6.18-prep（版本可查看/lib/modules目录）

安装reiserfsprogs-3.6.20.tar
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lvcreate -L 2048m -m1 -n storage-web dataVG /dev/mapper/mpath0 /dev/mapper/mpath1 /dev/mapper/mpath15 /dev/mapper/mpath16
mkreiserfs /dev/dataVG/storage-web

如何恢复镜像

1,首先创建丢失的PV，pvscan,检查PV是否可用
pvcreate /dev/...

2,将PV加入VG中，然后再PVSCAN检查一遍
vgextend dataVG /dev/...

3,对各个LV重新进行镜像操作
lvconvert -m1 /dev/dataVG/testLV /dev/mapper/mpath3 /dev/mapper/mpath4 /dev/mapper/mpath5

4,最后检查镜像是否完成
lvs -ao +devices

~~~~~~~~~~ ext3+lvm的扩展能力。
[root@samba multipath]# umount /mnt/web/
[root@samba multipath]# lvchange -an /dev/dataVG/test
[root@samba multipath]# lvextend -L+40G /dev/dataVG/test
  Extending 2 mirror images.
  Extending logical volume test to 150.00 GB
  Logical volume test successfully resized
[root@samba multipath]#
[root@samba multipath]# lvchange -ay /dev/dataVG/test
#扩展到跟LV一样大。
[root@samba multipath]# resize2fs /dev/vg0/lv0
[root@samba multipath]# mount /dev/dataVG/test /mnt/web/

example:
[root@samba mapper]# lvcreate -L 90g -m1 -n test2 dataVG /dev/mapper/mpath2 /dev/mapper/mpath16 /dev/mapper/mpath0 --nosync   
  WARNING: New mirror won't be synchronised. Don't read what you didn't write!
  Logical volume "test2" created
[root@samba mapper]# lvchange -an /dev/dataVG/test2
[root@samba mapper]# lvextend -L+100G /dev/dataVG/test2 /dev/mapper/mpath4 /dev/mpath/mpath5 /dev/mapper/mpath20 /dev/mapper/mpath21
  Extending 2 mirror images.
  Extending logical volume test2 to 190.00 GB
  Logical volume test2 successfully resized
[root@samba mapper]#
结果
2，4，20一个镜像，16，5，21一个镜像。