分类: LINUX
2012-09-05 18:44:47
MHVTL 是一个Linux下免费开源的虚拟带库模拟软件,可以模拟STK,IBM等多种带库。
详细介绍参见:
(可能需要fanqiang软件)
在本例中,有了MHVTL后,可以将大容量的磁盘模拟成IBM的磁带库,以供备份软件(如TSM,NBU等)使用。
为了让其他的机器能够访问这个MHVTL,还需要提供SCST (Generic SCSI Target Subsystem)。SCST允许在Linux上创建复杂的存储目标设备,这些目标设备可以提供给外界访问。它包括SCSI target Core和SCSI Target Drivers以及后端的设备处理模块(如scst_tape, scst_changer,scst_vdisk等)。
SCSI Target Drivers包括iSCSI, Fiber Channel, SAS, FCoE等协议的支持。
另外一个重要的组件是HBA卡(FC协议)的target mode driver,一般主机上的HBA卡是initiator mode的,可以访问别的设备,但是不能被别的主机访问。而一般被访问的设备上的HBA卡是target mode的,这样,被访问设备上(如磁盘阵列,磁带库等)的磁盘或是驱动器等就能被别的主机所访问。
这样本机上MHVTL虚拟带库,通过SCST将其包装成目标设备,再通过Target mode 的HBA光纤卡将其共享出去,就构成了一个简单的支持FC协议访问的虚拟带库。在本例中,由于使用的是Qlogic的HBA卡,所以需要安装QLogic的target driver,如果使用的是Emulex的HBA卡,那么有不同的方法安装Emulex光纤卡, 具体参见:
为了方便整个编译和安装的过程,先简单配置一下yum, 从而在安装一些依赖包时节省时间。
首先添加一个yum repository, 使用命令yum-config-manager:
#yum-config-manager --add-repo repository_url
例如:
# yum-config-manager --add-repo
或是添加本地光盘:
[root@mhvtlserver yum.repos.d]# yum-config-manager --add-repo file:///media/media.repo
Loaded plugins: product-id, refresh-packagekit
adding repo from: file:///media/media.repo
grabbing file file:///media/media.repo to /etc/yum.repos.d/media.repo
media.repo | 114 B 00:00 ...
repo saved to /etc/yum.repos.d/media.repo
[root@mhvtlserver yum.repos.d]# ls
media.repo packagekit-media.repo redhat.repo rhel-source.repo
修改本地光盘的描述media.repo,添加:
[InstallMedia]
baseurl=file:///media/ (/media是光盘所mount的目录)
..
列出所有repository:
[root@mhvtlserver yum.repos.d]# yum repolist all
Loaded plugins: product-id, refresh-packagekit, subscription-manager
Updating Red Hat repositories.
repo id repo name status
InstallMedia Red Hat Enterprise Linux 6.1 disabled
rhel-source Red Hat Enterprise Linux 6Server - x86_64 - Source disabled
rhel-source-beta Red Hat Enterprise Linux 6Server Beta - x86_64 – Sourc disabled
repolist: 0
因为目前 InstallMedia repository是disabled,所以需要enable 它:
[root@mhvtlserver yum.repos.d]# yum-config-manager --enable InstallMedia
Loaded plugins: product-id, refresh-packagekit
============================== repo: InstallMedia ==============================
[InstallMedia]
...
再列一下repository:
[root@mhvtlserver yum.repos.d]# yum repolist all
Loaded plugins: product-id, refresh-packagekit, subscription-manager
Updating Red Hat repositories.
repo id repo name status
InstallMedia Red Hat Enterprise Linux 6.1 enabled: 3,480
rhel-source Red Hat Enterprise Linux 6Server - x86_64 - Sour disabled
rhel-source-beta Red Hat Enterprise Linux 6Server Beta - x86_64 - disabled
repolist: 3,480
InstallMedia这个repository变成了enabled,可以使用了。
测试安装:
看需要的软件libXm.so.3(这个是部分名字即可,不一定是全名)是由那个可以安装的模块提供的:
[root@mhvtlserver ~]# yum whatprovides libXm.so.3
Loaded plugins: product-id, refresh-packagekit, subscription-manager
Updating Red Hat repositories.
openmotif22-2.2.3-19.el6.i686 : Open Motif runtime libraries and executables
Repo : InstallMedia
Matched from:
Other : libXm.so.3
安装openmotif22-2.2.3-19.el6.i686:
[root@mhvtlserver ~]# yum install openmotif22-2.2.3-19.el6.i686
…
首先使用yum 安装一些工具软件:
yum install zlib-devel
yum install lsscsi
yum install mt-st
yum install mtx
yum install lzo
还有其他包,可能需要从网上下载:
lzo-devel-2.03-3.1.el6.x86_64
lzo-minilzo-2.03-3.1.el6.x86_64
安装完毕后,从MHVTL网站下载所需的包:
mhvtl-2012-08-08.tgz
首先添加VTL用户:
#useradd vtl
#passwd vtl
#mkdir /opt/mhvtl (这个目录是存放虚拟磁带的海量存储空间)
#chown –Rf vtl:vtl /opt/mhvtl
创建mhvtl的配置目录:
mkdir /etc/mhvtl
chown -Rf vtl:vtl /etc/mhvtl(这个目录是存放mhvtl的存储空间)
MHVTL INSTALL:
# tar xvf mhvtl-2012-08-08.tgz
#cd mhvtl-1.4/
# make distclean
首先安装kernel部分:
# cd kernel/
# make distclean
#make
# make install
然后其他部分:
# cd ..
#make
#make install
安装完成后,/etc/init.d/mhvtl start即可启动mhvtl, 如果没有配置文件,会自动生成缺省的配置文件。
停止mhvtl:
#/etc/init.d/mhvtl stop
停止mhvtl后,为了便于其他的备份软件使用,编辑修改mhvtl配置文件,主要包括device.conf,mhvtl.conf, library_contents.xx ,如下:
[root@mhvtlserver mhvtl]# more /etc/mhvtl/device.conf
VERSION: 5
# VPD page format:
#
# NAA format is an 8 hex byte value seperated by ':'
# Note: NAA is part of inquiry VPD 0x83
#
# Each 'record' is separated by one (or more) blank lines.
# Each 'record' starts at column 1
# Serial num max len is 10.
# Compression: factor X enabled 0|1
# Where X is zlib compression factor 1 = Fastest compression
# 9 = Best compression
# enabled 0 == off, 1 == on
#
# fifo: /var/tmp/mhvtl
# If enabled, data must be read from fifo, otherwise daemon will block
# trying to write.
# e.g. cat /var/tmp/mhvtl (in another terminal)
Library: 10 CHANNEL: 00 TARGET: 00 LUN: 00 (定义带库,这一行中的10是带库的标识,带库的配置则在相应的library_contents.10文件中指定)
Vendor identification: IBM
Product identification: 03584L32
Product revision level: 4.02
Unit serial number: 9890400
NAA: 50:22:33:44:ab:00:00:00
Home directory: /opt/mhvtl
# fifo: /var/tmp/mhvtl
Drive: 11 CHANNEL: 00 TARGET: 01 LUN: 00 (以下定义各个驱动器)
Library ID: 10 Slot: 01
Vendor identification: IBM
Product identification: ULTRIUM-TD3
Product revision level: 51TB
Unit serial number: 9825111
NAA: 10:22:33:44:ab:00:01:00
Compression: factor 1 enabled 1
Compression type: lzo
Backoff: 1000
# fifo: /var/tmp/mhvtl
Drive: 12 CHANNEL: 00 TARGET: 02 LUN: 00
Library ID: 10 Slot: 02
Vendor identification: IBM
Product identification: ULTRIUM-TD3
Unit serial number: 9825112
Product revision level: 51TB
NAA: 10:22:33:44:ab:00:02:00
Compression: factor 1 enabled 1
Compression type: lzo
Backoff: 1000
# fifo: /var/tmp/mhvtl
Drive: 13 CHANNEL: 00 TARGET: 03 LUN: 00
Library ID: 10 Slot: 03
Vendor identification: IBM
Product identification: ULTRIUM-TD3
Unit serial number: 9825113
Product revision level: 51TB
NAA: 10:22:33:44:ab:00:03:00
Compression: factor 1 enabled 1
Compression type: lzo
Backoff: 1000
# fifo: /var/tmp/mhvtl
Drive: 14 CHANNEL: 00 TARGET: 04 LUN: 00
Library ID: 10 Slot: 04
Vendor identification: IBM
Product identification: ULTRIUM-TD3
Unit serial number: 9825114
Product revision level: 51TB
NAA: 10:22:33:44:ab:00:04:00
Compression: factor 1 enabled 1
Compression type: lzo
Backoff: 1000
# fifo: /var/tmp/mhvtl
Library_contents.10中指定磁带,驱动器等的数量,这里磁带类型用L3---因为驱动器是ULTRIUM-TD3:
[root@mhvtlserver mhvtl]# more library_contents.10
Drive 1:
Drive 2:
Drive 3:
Drive 4:
Picker 1:
MAP 1:
MAP 2:
MAP 3:
MAP 4:
# Slot 1 - ?, no gaps
# Slot N: [barcode]
# [barcode]
# a barcode is comprised of three fields: [Leading] [identifier] [Trailing]
# Leading "CLN" -- cleaning tape
# Leading "W" -- WORM tape
# Leading "NOBAR" -- will appear to have no barcode
# If the barcode is at least 8 character long, then the last two characters are Trailing
# Trailing "S3" - SDLT600
# Trailing "X4" - AIT-4
# Trailing "L1" - LTO 1, "L2" - LTO 2, "L3" - LTO 3, "L4" - LTO 4, "L5" - LTO 5
# Training "LT" - LTO 3 WORM, "LU" LTO 4 WORM, "LV" LTO 5 WORM
# Trailing "TA" - T10000+
# Trailing "JA" - 3592+
# Trailing "JB" - 3592E05+
# Trailing "JW" - WORM 3592+
# Trailing "JX" - WORM 3592E05+
#
Slot 1: E01001L3
Slot 2: E01002L3
Slot 3: E01003L3
Slot 4: E01004L3
Slot 5: E01005L3
Slot 6: E01006L3
Slot 7: E01007L3
Slot 8: E01008L3
Slot 9: E01009L3
Slot 10: E01010L3
Slot 11: E01011L3
Slot 12: E01012L3
Slot 13: E01013L3
Slot 14: E01014L3
Slot 15: E01015L3
Slot 16: E01016L3
Slot 17: E01017L3
Slot 18: E01018L3
Slot 19: E01019L3
Slot 20: E01020L3
Slot 21:
Slot 22:
Slot 23:
Slot 24:
Slot 25:
Slot 26:
Slot 27:
Slot 28:
Slot 29:
Slot 30: E01030L3
Slot 31: E01031L3
Slot 32: E01032L3
Slot 33: E01033L3
Slot 34: E01034L3
Slot 35: E01035L3
Slot 36: E01036L3
Slot 37: E01037L3
Slot 38: E01038L3
Slot 39: E01039L3
mhvtl.conf配置文件:
[root@mhvtlserver mhvtl]# more /etc/mhvtl/mhvtl.conf
# Home directory for config file(s)
MHVTL_CONFIG_PATH=/etc/mhvtl
# Default media capacity (500 M)
CAPACITY=400000 (每盘磁带400G)
# Set default verbosity [0|1|2|3]
VERBOSE=1
# Set kernel module debuging [0|1]
VTL_DEBUG=0
修改完毕mhvtl的配置文件后,启动mhvtl:
#/etc/init.d/mhvtl start
将其添加到chkconfig中:
#chkconfig --add mhvtl
#chkconfig mhvtl {on|off} (设置自动启动,这里设置为off,然后在/etc/rc.local中启动)
查看生成的mhvtl设备:
#lsscsi -g
…
[6:0:0:0] mediumx IBM 03584L32 4.02 - /dev/sg21
[6:0:1:0] tape IBM ULTRIUM-TD3 51TB - /dev/sg17
[6:0:2:0] tape IBM ULTRIUM-TD3 51TB - /dev/sg18
[6:0:3:0] tape IBM ULTRIUM-TD3 51TB - /dev/sg19
[6:0:4:0] tape IBM ULTRIUM-TD3 51TB - /dev/sg20
[root@mhvtlserver mhvtl]# mtx -f /dev/sg21 status (/dev/sg21为03584L32 mediumx)
Storage Changer /dev/sg21:4 Drives, 43 Slots ( 4 Import/Export )
Data Transfer Element 0:Empty
Data Transfer Element 1:Empty
Data Transfer Element 2:Empty
Data Transfer Element 3:Empty
Storage Element 1:Full :VolumeTag=E01001L3
Storage Element 2:Full :VolumeTag=E01002L3
Storage Element 3:Full :VolumeTag=E01003L3
Storage Element 4:Full :VolumeTag=E01004L3
Storage Element 5:Full :VolumeTag=E01005L3
Storage Element 6:Full :VolumeTag=E01006L3
Storage Element 7:Full :VolumeTag=E01007L3
Storage Element 8:Full :VolumeTag=E01008L3
Storage Element 9:Full :VolumeTag=E01009L3
Storage Element 10:Full :VolumeTag=E01010L3
Storage Element 11:Full :VolumeTag=E01011L3
Storage Element 12:Full :VolumeTag=E01012L3
Storage Element 13:Full :VolumeTag=E01013L3
Storage Element 14:Full :VolumeTag=E01014L3
Storage Element 15:Full :VolumeTag=E01015L3
Storage Element 16:Full :VolumeTag=E01016L3
Storage Element 17:Full :VolumeTag=E01017L3
Storage Element 18:Full :VolumeTag=E01018L3
Storage Element 19:Full :VolumeTag=E01019L3
Storage Element 20:Full :VolumeTag=E01020L3
Storage Element 21:Empty
Storage Element 22:Empty
Storage Element 23:Empty
Storage Element 24:Empty
Storage Element 25:Empty
Storage Element 26:Empty
Storage Element 27:Empty
Storage Element 28:Empty
Storage Element 29:Empty
Storage Element 30:Full :VolumeTag=E01030L3
Storage Element 31:Full :VolumeTag=E01031L3
Storage Element 32:Full :VolumeTag=E01032L3
Storage Element 33:Full :VolumeTag=E01033L3
Storage Element 34:Full :VolumeTag=E01034L3
Storage Element 35:Full :VolumeTag=E01035L3
Storage Element 36:Full :VolumeTag=E01036L3
Storage Element 37:Full :VolumeTag=E01037L3
Storage Element 38:Full :VolumeTag=E01038L3
Storage Element 39:Full :VolumeTag=E01039L3
Storage Element 40 IMPORT/EXPORT:Empty
Storage Element 41 IMPORT/EXPORT:Empty
Storage Element 42 IMPORT/EXPORT:Empty
Storage Element 43 IMPORT/EXPORT:Empty
将磁带从1号槽位装入driver 0:
[root@mhvtlserver mhvtl]# mtx -f /dev/sg21 load 1 0
Loading media from Storage Element 1 into drive 0...done
[root@mhvtlserver mhvtl]# mtx -f /dev/sg21 status
Storage Changer /dev/sg21:4 Drives, 43 Slots ( 4 Import/Export )
Data Transfer Element 0:Full (Storage Element 1 Loaded):VolumeTag = E01001L3
Data Transfer Element 1:Empty
Data Transfer Element 2:Empty
Data Transfer Element 3:Empty
Storage Element 1:Empty
Storage Element 2:Full :VolumeTag=E01002L3
…
将磁带从drive0卸到1号槽位:
[root@mhvtlserver mhvtl]# mtx -f /dev/sg21 unload 1 0
Unloading drive 0 into Storage Element 1...done
[root@mhvtlserver mhvtl]# mtx -f /dev/sg21 status
Storage Changer /dev/sg21:4 Drives, 43 Slots ( 4 Import/Export )
Data Transfer Element 0:Empty
Data Transfer Element 1:Empty
Data Transfer Element 2:Empty
Data Transfer Element 3:Empty
Storage Element 1:Full :VolumeTag=E01001L3
…
读写测试:
载入39号槽位的磁带到drive 0 (/dev/IBMtape0)并读写:
[root@mhvtlserver mhvtl]# mtx -f /dev/sg21 load 39 0
Loading media from Storage Element 39 into drive 0...done
[root@mhvtlserver mhvtl]# cd /opt/mhvtl/E01039L3/
[root@mhvtlserver E01039L3]# ls -l
total 4
-rw-rw---- 1 vtl vtl 0 Aug 14 15:43 data
-rw-rw---- 1 vtl vtl 0 Aug 14 15:43 indx
-rw-rw---- 1 vtl vtl 1536 Aug 29 17:55 meta
查看
[root@mhvtlserver E01039L3]# mt -f /dev/IBMtape0 status
SCSI 2 tape drive:
File number=-1, block number=0, partition=0.
Tape block size 0 bytes. Density code 0x44 (LTO-3).
Soft error count since last status=0
General status bits on (41000000):
BOT ONLINE
查看没有磁带的驱动器:
[root@mhvtlserver E01039L3]# mt -f /dev/IBMtape1 status
SCSI 2 tape drive:
File number=-1, block number=-1, partition=0.
Tape block size 0 bytes. Density code 0x44 (LTO-3).
Soft error count since last status=0
General status bits on (40000):
DR_OPEN
写入磁带:
[root@mhvtlserver E01039L3]# tar cvvf /dev/IBMtape0 /tmp/software/RHEL6.1X86_64.iso
tar: Removing leading `/' from member names
-rw-r--r-- root/root 3600842752 2012-08-16 16:33 /tmp/software/RHEL6.1X86_64.iso
[root@mhvtlserver E01039L3]# ls -l
total 3617140
-rw-rw---- 1 vtl vtl 3520277200 Aug 29 18:20 data
-rw-rw---- 1 vtl vtl 180043264 Aug 29 18:20 indx
-rw-rw---- 1 vtl vtl 1544 Aug 29 18:20 meta
读:(列出磁带内容)
[root@mhvtlserver E01039L3]# mt -f /dev/IBMtape0 rewind
[root@mhvtlserver E01039L3]# tar tvf /dev/IBMtape0
-rw-r--r-- root/root 3600842752 2012-08-16 16:33 tmp/software/RHEL6.1X86_64.iso
擦掉磁带内容再查看:
[root@mhvtlserver E01039L3]# mt -f /dev/IBMtape0 erase
[root@mhvtlserver E01039L3]# tar tvf /dev/IBMtape0
tar: /dev/IBMtape0: Cannot read: Input/output error
tar: At beginning of tape, quitting now
tar: Error is not recoverable: exiting now
[root@mhvtlserver E01039L3]# ls -l
total 4
-rw-rw---- 1 vtl vtl 0 Aug 29 18:23 data
-rw-rw---- 1 vtl vtl 0 Aug 29 18:23 indx
-rw-rw---- 1 vtl vtl 1536 Aug 29 18:23 meta
[root@mhvtlserver E01039L3]# mt -f /dev/IBMtape0 rewind
[root@mhvtlserver E01039L3]# mt -f /dev/IBMtape0 status
SCSI 2 tape drive:
File number=-1, block number=0, partition=0.
Tape block size 0 bytes. Density code 0x44 (LTO-3).
Soft error count since last status=0
General status bits on (41000000):
BOT ONLINE
将该磁带放回39号槽位:
[root@mhvtlserver E01039L3]# mtx -f /dev/sg21 unload 39 0
Unloading drive 0 into Storage Element 39...done
[root@mhvtlserver E01039L3]# mtx -f /dev/sg21 status
Storage Changer /dev/sg21:4 Drives, 43 Slots ( 4 Import/Export )
Data Transfer Element 0:Empty
Data Transfer Element 1:Empty
Data Transfer Element 2:Empty
Data Transfer Element 3:Empty
…
Storage Element 39:Full :VolumeTag=E01039L3
…
本机上的MHVTL配置完成并测试通过后,需要配置SCST和Qlogic HBA卡的target mode driver。
安装openssl ,openssl-devel等软件:
#yum install openssl openssl-devel gcc ncurses-devel kernel-devel lsscsi patch subversion
从Qlogic网站或是下列地址:
下载最新Firmware,并保存到/lib/firmware,否则在使用Qlogic HBA卡时可能会报错:
…
qla2xxx 0000:13:00.0: Firmware image unavailable.
qla2xxx 0000:13:00.0: Failed to initialize adapter
…
本例中使用的是:
ql2400_fw.bin
从 下载scst, scstadmin, iscsi-scst, qla2x00t的源代码:
iscsi-scst-2.2.0.tar
qla2x00t-2.2.0.tar
scst-2.2.0.tar
scstadmin-2.2.0.tar
并使用
#tar xvf xxxx.tar 解压缩各个tar包。
如果没有mockbuild用户那么使用useradd 添加mockbuild用户,用来编译安装内核代码。
移去当前正在使用的由系统提供的qla2xxx模块,并在下次启动时屏蔽:
#rmmod qla2xxx
#echo blacklist qla2xxx >/etc/modprobe.d/blacklist-qla2xxx.conf
#reboot
安装Kernel Source:
KERNEL BUILD:
[root@mhvtlserver ~]# cd ~
[root@mhvtlserver ~]# rm -fr rpmbuild/
[root@mhvtlserver ~]# cd Downloads/
[root@mhvtlserver Downloads]# ls
..
kernel-2.6.32-131.0.15.el6.src.rpm
qla2x00t-2.2.0.tar
..
[root@mhvtlserver Downloads]# rpm -ivh kernel-2.6.32-131.0.15.el6.src.rpm
warning: kernel-2.6.32-131.0.15.el6.src.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
1:kernel ########################################### [100%]内核源代码安装完成后,开始做一些修改,然后再重新编译内核:
[root@mhvtlserver Downloads]# cd /root
[root@mhvtlserver ~]# cd rpmbuild/ (RHEL6.1的rpmbuild放在这个目录下)
[root@mhvtlserver rpmbuild]# ls
SOURCES SPECS
[root@mhvtlserver rpmbuild]# cd SPECS/
[root@mhvtlserver SPECS]# ls
kernel.spec
[root@mhvtlserver SPECS]# rngd -r /dev/urandom (生成随机数,否则下面的rpmbuild的过程会很漫长)[root@mhvtlserver rpmbuild]# mkdir -p BUILD
[root@mhvtlserver rpmbuild]# chown mockbuild BUILD执行kernel.spec的"%prep"部分,主要是解开源代码并打相应补丁:
[root@mhvtlserver SPECS]# rpmbuild -bp kernel.spec到包含qla2xxx驱动模块的目录下:
[root@mhvtlserver rpmbuild]# cd /root/rpmbuild/BUILD
[root@mhvtlserver BUILD]# cd kernel-2.6.32-131.0.15.el6/
[root@mhvtlserver kernel-2.6.32-131.0.15.el6]# cd linux-2.6.32-131.0.15.el6.x86_64/
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]# pwd
/root/rpmbuild/BUILD/kernel-2.6.32-131.0.15.el6/linux-2.6.32-131.0.15.el6.x86_64
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]# cd drivers/scsi/将原有的kernel source自带的qla2xxx目录改名保存:
[root@mhvtlserver scsi]# mv qla2xxx/ qla2xxx_orig将解压缩的qla2x00t-2.2.0改名,并建立链接,也就是用下载的qla2x00t-2.2.0中的qla2xxx驱动替代原有的kenel source自带的qla2xxx驱动:
[root@mhvtlserver scsi]#mv /root/Downloads/qla2x00t-2.2.0 /root/Downloads/qla2x00t
[root@mhvtlserver scsi]# ln -s /root/Downloads/qla2x00t/ qla2xxx然后对现有的kernel source打补丁,包括两个补丁,这两个补丁来分别自两个不同的包,它们是scst-2.2.0.tar和iscsi-scst-2.2.0.tar:
[root@mhvtlserver scsi]# cd ../..
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]# pwd
/root/rpmbuild/BUILD/kernel-2.6.32-131.0.15.el6/linux-2.6.32-131.0.15.el6.x86_64
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]# patch -p1 < /root/Downloads/scst-2.2.0/kernel/rhel/scst_exec_req_fifo-2.6.32.patch
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]# patch -p1 < /root/Downloads/iscsi-scst-2.2.0/kernel/patches/rhel/put_page_callback-2.6.32-131.patch使用GUI配置内核,这个比字符方式要简单多了:
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]# make xconfig
选择菜单:
"Device Drivers -> SCSI device support -> SCSI low-level drivers"
确保”QLogic QLA2XXX Fibre Channel Support -> QLogic 2xxx target mode support”
被选中。然后,移动到"SCSI debugging host simulator" 并选中,
保存退出。修改内核Makefile,加上内核扩展名:
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]#vi Makefile
EXTRAVERSION = -scst开始编译内核:
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]# make
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]# make modules; mkdir -p /lib/modules/2.6.18-scst
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]#make modules_install && make install编译完毕后重启,选择新内核启动:
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]#reboot (select the '-scst' kernel.)这样,在编译内核的过程中,新的QLogic 的 driver qla2xxx就顺带被编译并加入到了新内核中,新内核中的Qlogic HBA卡的驱动此时是支持initiator/target mode的,这个可以使用
#modinfo qla2xxx
查看,其中有qlini_mode参数,允许选择target mode.
[root@mhvtlserver scst-2.2.0]# cd /root/Downloads/scst-2.2.0
生成SCST Release版本,只有很少的告警信息,没有调试信息,然后安装:
[root@mhvtlserver scst-2.2.0]# make debug2release
[root@mhvtlserver scst-2.2.0]# cd /root/Downloads/scst-2.2.0/src[root@mhvtlserver src]# vi Makefile
Uncomment: EXTRA_CFLAGS += -DCONFIG_SCST_STRICT_SERIALIZING
(Needed for tape emulation)
[root@mhvtlserver scst-2.2.0]# cd /root/Downloads/scst-2.2.0
[root@mhvtlserver scst-2.2.0]# make
[root@mhvtlserver scst-2.2.0]# make install之后,在/lib/modules/2.6.32-scst/extra下会有:
..
-rw-r--r-- 1 root root 3734064 Aug 20 12:28 scst.ko
..
[root@mhvtlserver dev_handlers]# pwd
/lib/modules/2.6.32-scst/extra/dev_handlers
[root@mhvtlserver dev_handlers]# ls -l
total 3380
-rw-r--r-- 1 root root 306372 Aug 20 12:28 scst_cdrom.ko
-rw-r--r-- 1 root root 288539 Aug 20 12:28 scst_changer.ko
-rw-r--r-- 1 root root 344607 Aug 20 12:28 scst_disk.ko
-rw-r--r-- 1 root root 311477 Aug 20 12:28 scst_modisk.ko
-rw-r--r-- 1 root root 288563 Aug 20 12:28 scst_processor.ko
-rw-r--r-- 1 root root 288499 Aug 20 12:28 scst_raid.ko
-rw-r--r-- 1 root root 313697 Aug 20 12:28 scst_tape.ko
-rw-r--r-- 1 root root 624139 Aug 20 12:28 scst_user.ko
-rw-r--r-- 1 root root 672542 Aug 20 12:28 scst_vdisk.ko说明:
scst - SCST itself.
scst_disk - device handler for disks (type 0).
scst_tape - device handler for tapes (type 1).
scst_processor - device handler for processors (type 3).
scst_cdrom - device handler for CDROMs (type 5).
scst_modisk - device handler for MO disks (type 7).
scst_changer - device handler for medium changers (type 8).
scst_raid - device handler for storage array controller (e.g. raid) (type C).
scst_vdisk - device handler for virtual disks (file, device or ISO CD image)..
scst_user - user space device handler.在/usr/local/include/scst/下也会有一些文件。
下一步,生成Qlogic target mode driver add-on,并安装:
[root@mhvtlserver scst-2.2.0]#cd /root/Downloads/qla2x00t/qla2x00-target修改SCST include和src文件的路径,否则会报错:
[root@mhvtlserver src]# vi Makefile
…
#SCST_INC_DIR := /usr/local/include/scst
#SCST_DIR := $(SCST_INC_DIR)
SCST_INC_DIR := /root/Downloads/scst-2.2.0/include
SCST_DIR := /root/Downloads/scst-2.2.0/src
…[root@mhvtlserver qla2x00-target]# make clean
[root@mhvtlserver qla2x00-target]# make
[root@mhvtlserver qla2x00-target]# make uninstall
[root@mhvtlserver qla2x00-target]# make install编译完成后,在/lib/modules/2.6.32-scst/extra下生成:
-rw-r--r-- 1 root root 982816 Aug 20 12:38 qla2x00tgt.ko
手动加载这些模块:
#modprobe scst
#modprobe scst_changer
#modprobe scst_tape应该会生成/sys/kernel/scst_tgt目录:
[root@mhvtlserver scst_tgt]# ls -F
device_groups/ handlers/ max_tasklet_cmd sgv/ threads version
devices/ last_sysfs_mgmt_res setup_id targets/ trace_levelhandlers目录下应该有:
[root@mhvtlserver scst_tgt]# ls -F handlers/
dev_changer/ dev_tape/ dev_tape_perf/ ...
dmesg看应该有:
Aug 6 13:20:44 mhvtlserver kernel: [2885]: scst: init_scst:2362:SCST version 2.2.0 loaded successfully (max mem for commands 1964MB, per device 785MB)
Aug 6 13:20:44 mhvtlserver kernel: [2916]: scst: __scst_register_dev_driver:1406:Device handler "dev_tape" for type 1 registered successfully
Aug 6 13:20:44 mhvtlserver kernel: [2916]: scst: __scst_register_dev_driver:1406:Device handler "dev_tape_perf" for type 1 registered successfully
Aug 6 13:20:44 mhvtlserver kernel: [2923]: scst: __scst_register_dev_driver:1406:Device handler "dev_changer" for type 8 registered successfully
…然后加载qla2xxx和qla2x00tgt:
首先修改modprobe配置,去掉blacklist-qla2xxx.conf,然后添加qla.conf:
[root@mhvtlserver modprobe.d]# more /etc/modprobe.d/qla.conf
options qla2xxx qlini_mode="disabled"这里qlini_mode说明如下:
When qla2xxx compiled with CONFIG_SCSI_QLA2XXX_TARGET enabled, it has
parameter "qlini_mode", which determines when initiator mode will be
enabled. Possible values:
- "exclusive" (default) - initiator mode will be enabled on load,
disabled on enabling target mode and then on disabling target mode
enabled back.
- "disabled" - initiator mode will never be enabled.
- "enabled" - initiator mode will always stay enabled.加载这些模块:
#modprobe qla2xxx
#modprobe qla2x00tgtdmesg:
这个是qla2xxx:
Aug 6 13:20:32 mhvtlserver kernel: QLogic Fibre Channel HBA Driver: 8.02.01-k4-tgt
Aug 6 13:20:32 mhvtlserver kernel: qla2xxx 0000:05:00.0: PCI INT A -> GSI 35 (level, low) -> IRQ 35
Aug 6 13:20:32 mhvtlserver kernel: qla2xxx 0000:05:00.0: Found an ISP2432, irq 35, iobase 0xffffc90011836000
Aug 6 13:20:32 mhvtlserver kernel: qla2xxx 0000:05:00.0: Configuring PCI space...
Aug 6 13:20:32 mhvtlserver kernel: qla2xxx 0000:05:00.0: Configure NVRAM parameters...
Aug 6 13:20:32 mhvtlserver kernel: qla2xxx 0000:05:00.0: Verifying loaded RISC code...
Aug 6 13:20:32 mhvtlserver kernel: qla2xxx 0000:05:00.0: firmware: requesting ql2400_fw.bin
Aug 6 13:20:32 mhvtlserver kernel: scsi5 : qla2xxx
Aug 6 13:20:32 mhvtlserver kernel: qla2xxx 0000:05:00.0:
Aug 6 13:20:32 mhvtlserver kernel: QLogic Fibre Channel HBA Driver: 8.02.01-k4-tgt
…qla2x00tgt:
Aug 20 12:34:03 mhvtlserver kernel: [17037]: q2t_init:6546:qla2x00t: Initializing QLogic Fibre Channel HBA Driver target mode addon version 2.2.0
Aug 20 12:34:03 mhvtlserver kernel: scst: Target 21:00:00:24:ff:05:24:b4 for template qla2x00t registered successfully
Aug 20 12:34:03 mhvtlserver kernel: [17037]: q2t_add_target:5758:qla2x00t(0): using 64 Bit PCI addressing
Aug 20 12:34:03 mhvtlserver kernel: [17037]: q2t_target_detect:602:qla2x00t: Target mode driver for QLogic 2x00 controller registered successfully
Aug 20 12:34:03 mhvtlserver kernel: scst: Target template qla2x00t registered successfully
…此时,应该可以在其他的主机上能够看到这些被输出的设备。(客户机可能需要重新启动,或是重新扫描硬件)
在另外一台机器上:
#dmesg
…
scsi 2:0:2:0: Medium Changer IBM 03584L32 4.02 PQ: 0 ANSI: 5
ch0: type #1 (mt): 0x100+1 [medium transport]
ch0: type #2 (st): 0x400+39 [storage]
ch0: type #3 (ie): 0x200+4 [import/export]
ch0: type #4 (dt): 0x1+4 [data transfer]
ch0: dt 0x1: ID/LUN unknown
ch0: dt 0x2: ID/LUN unknown
ch0: dt 0x3: ID/LUN unknown
ch0: dt 0x4: ID/LUN unknown
ch0: INITIALIZE ELEMENT STATUS, may take some time ...
ch0: ... finished
ch 2:0:2:0: Attached scsi changer ch0
ch 2:0:2:0: Attached scsi generic sg13 type 8
lin_tape: detected device at host 2, channel 0, id 2, lun 0
IBM 03584L32 4.02
lin_tape: attached IBMchanger0
scsi 2:0:2:1: Sequential-Access IBM ULTRIUM-TD3 51TB PQ: 0 ANSI: 5
lin_tape 2:0:2:1: Attached scsi generic sg14 type 1
lin_tape: detected device at host 2, channel 0, id 2, lun 1
IBM ULTRIUM-TD3 51TB
lin_tape: attached IBMtape0
scsi 2:0:2:2: Sequential-Access IBM ULTRIUM-TD3 51TB PQ: 0 ANSI: 5
lin_tape 2:0:2:2: Attached scsi generic sg15 type 1
lin_tape: detected device at host 2, channel 0, id 2, lun 2
IBM ULTRIUM-TD3 51TB
lin_tape: attached IBMtape1
scsi 2:0:2:3: Sequential-Access IBM ULTRIUM-TD3 51TB PQ: 0 ANSI: 5
lin_tape 2:0:2:3: Attached scsi generic sg16 type 1
lin_tape: detected device at host 2, channel 0, id 2, lun 3
IBM ULTRIUM-TD3 51TB
lin_tape: attached IBMtape2
scsi 2:0:2:4: Sequential-Access IBM ULTRIUM-TD3 51TB PQ: 0 ANSI: 5
lin_tape 2:0:2:4: Attached scsi generic sg17 type 1
lin_tape: detected device at host 2, channel 0, id 2, lun 4
IBM ULTRIUM-TD3 51TB
lin_tape: attached IBMtape3
…
修改/etc/init.d/scst文件,自动加载scst scst_changer scst_tape qla2x00tgt等驱动模块:
找到:
parse_scst_conf() {
SCST_MODULES=…
修改为:
parse_scst_conf() {
SCST_MODULES="scst scst_tape scst_changer scst_disk scst_vdisk qla2x00tgt" (scst_disk,scst_vdisk不用的话可以去掉)通过lsscsi的结果创建/etc/scst.conf文件,可以用脚本完成:
[root@mhvtlserver init.d]# more /usr/local/bin/crscstconf.sh
# Build /etc/scst.conf
# Header of scst.conf
echo "# Header Info" >/tmp/scst.tmp
echo "setup_id 0x1234" >>/tmp/scst.tmp
echo "# Changer Devices " >>/tmp/scst.tmp# define dev_changer
echo "HANDLER dev_changer {" >>/tmp/scst.tmp
lsscsi -g| grep mediumx | awk '{print $1}'| cut -d "[" -f2| cut -d "]" -f1| while read each1; do
echo " DEVICE $each1" >>/tmp/scst.tmp
done
echo "}" >>/tmp/scst.tmp# define dev_tape
echo "# Tape device" >>/tmp/scst.tmp
echo "HANDLER dev_tape {" >>/tmp/scst.tmp
lsscsi -g| grep tape | awk '{print $1}'| cut -d "[" -f2| cut -d "]" -f1 | while read each2; do
echo " DEVICE $each2" >>/tmp/scst.tmp
done
echo "}" >>/tmp/scst.tmp# define Qlogic part.
echo "# Qlogic Devices " >>/tmp/scst.tmp
echo "TARGET_DRIVER qla2x00t {" >>/tmp/scst.tmp
echo " TARGET 21:00:00:24:ff:05:24:b4 {" >>/tmp/scst.tmpint=0
lsscsi -g | grep -e tape -e mediumx|awk '{print $1}'| cut -d "[" -f2| cut -d "]" -f1| while read each3; do
echo " LUN $int $each3" >>/tmp/scst.tmp
int=`expr $int + 1`
done
echo " enabled 1" >>/tmp/scst.tmp
echo " }" >>/tmp/scst.tmp
echo "}" >>/tmp/scst.tmpif [ -f /etc/scst.conf ]; then
cp -f /etc/scst.conf /etc/scst.conf_`date +%m%d%y%H%M%S`
echo Created backup of existing /etc/scst.conf as /etc/scst.conf_`date +%m%d%y%H%M%S`
fi
cat /tmp/scst.tmp >/etc/scst.conf
echo Created new /etc/scst.conf
rm -f /tmp/scst.tmpecho --- Done ---
这个脚本中的TARGET 21:00:00:24:ff:05:24:b4等需要从/sys/kernel/scst_tgt/targets目录下得到:
[root@mhvtlserver targets]# pwd
/sys/kernel/scst_tgt/targets
[root@mhvtlserver targets]# ls
qla2x00t
[root@mhvtlserver targets]# cd qla2x00t/
[root@mhvtlserver qla2x00t]# ls
21:00:00:24:ff:05:24:b4 mgmt trace_level version
[root@mhvtlserver qla2x00t]#执行这个脚本,生成:
[root@mhvtlserver qla2x00t]# more /etc/scst.conf
# Header Info
setup_id 0x1234
# Changer Devices
HANDLER dev_changer {
DEVICE 6:0:0:0
}
# Tape device
HANDLER dev_tape {
DEVICE 6:0:1:0
DEVICE 6:0:2:0
DEVICE 6:0:3:0
DEVICE 6:0:4:0
}
# Qlogic Devices
TARGET_DRIVER qla2x00t {
TARGET 21:00:00:24:ff:05:24:b4 {
LUN 0 6:0:0:0
LUN 1 6:0:1:0
LUN 2 6:0:2:0
LUN 3 6:0:3:0
LUN 4 6:0:4:0
enabled 1
}
}然后,启动scst, 其他的机器就只能看到这个/etc/scst.conf输出的设备:
#service scst stop
#service scst start修改/etc/rc.local,改为自动启动:
[root@mhvtlserver qla2x00t]# more /etc/rc.local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.touch /var/lock/subsys/local
# First, create the config file /etc/scst.conf for scst.
/usr/local/bin/crscstconf.sh
# Then start scst.
service scst start这样系统启动完毕后,会自动加载qla2xxx(由内核加载), scst scst_changer scst_tape qla2x00tgt等模块,并根据刚刚生成的/etc/scst.conf来配置那些需要输出的设备。