Chinaunix首页 | 论坛 | 博客
  • 博客访问: 174305
  • 博文数量: 17
  • 博客积分: 299
  • 博客等级: 下士
  • 技术积分: 292
  • 用 户 组: 普通用户
  • 注册时间: 2011-03-24 12:11
文章分类
文章存档

2017年(1)

2013年(1)

2012年(15)

我的朋友

分类: LINUX

2012-09-05 18:44:47

 

MHVTL 是一个Linux下免费开源的虚拟带库模拟软件,可以模拟STK,IBM等多种带库。
详细介绍参见:
(可能需要fanqiang软件)

在本例中,有了MHVTL后,可以将大容量的磁盘模拟成IBM的磁带库,以供备份软件(如TSM,NBU等)使用。

为了让其他的机器能够访问这个MHVTL,还需要提供SCST (Generic SCSI Target Subsystem)。SCST允许在Linux上创建复杂的存储目标设备,这些目标设备可以提供给外界访问。它包括SCSI target Core和SCSI Target Drivers以及后端的设备处理模块(如scst_tape, scst_changer,scst_vdisk等)。
SCSI Target Drivers包括iSCSI, Fiber Channel, SAS, FCoE等协议的支持。

另外一个重要的组件是HBA卡(FC协议)的target mode driver,一般主机上的HBA卡是initiator mode的,可以访问别的设备,但是不能被别的主机访问。而一般被访问的设备上的HBA卡是target mode的,这样,被访问设备上(如磁盘阵列,磁带库等)的磁盘或是驱动器等就能被别的主机所访问。

     这样本机上MHVTL虚拟带库,通过SCST将其包装成目标设备,再通过Target mode 的HBA光纤卡将其共享出去,就构成了一个简单的支持FC协议访问的虚拟带库。在本例中,由于使用的是Qlogic的HBA卡,所以需要安装QLogic的target driver,如果使用的是Emulex的HBA卡,那么有不同的方法安装Emulex光纤卡,    具体参见:



为了方便整个编译和安装的过程,先简单配置一下yum, 从而在安装一些依赖包时节省时间。

  • 2简单配置和测试

 

首先添加一个yum  repository, 使用命令yum-config-manager:

#yum-config-manager  --add-repo  repository_url

例如:
# yum-config-manager  --add-repo 

或是添加本地光盘:

[root@mhvtlserver yum.repos.d]# yum-config-manager  --add-repo  file:///media/media.repo
Loaded plugins: product-id, refresh-packagekit
adding repo from: file:///media/media.repo
grabbing file file:///media/media.repo to /etc/yum.repos.d/media.repo
media.repo                                               |  114 B     00:00 ...
repo saved to /etc/yum.repos.d/media.repo

[root@mhvtlserver yum.repos.d]# ls
media.repo  packagekit-media.repo  redhat.repo  rhel-source.repo

修改本地光盘的描述media.repo,添加:
[InstallMedia]
baseurl=file:///media/  (/media是光盘所mount的目录)
..

列出所有repository:
[root@mhvtlserver yum.repos.d]# yum  repolist  all
Loaded plugins: product-id, refresh-packagekit, subscription-manager
Updating Red Hat repositories.
repo id          repo name                                              status
InstallMedia     Red Hat Enterprise Linux 6.1                          disabled
rhel-source      Red Hat Enterprise Linux 6Server - x86_64 - Source         disabled
rhel-source-beta Red Hat Enterprise Linux 6Server Beta - x86_64 – Sourc     disabled
repolist: 0

因为目前 InstallMedia repository是disabled,所以需要enable 它:
[root@mhvtlserver yum.repos.d]# yum-config-manager  --enable  InstallMedia
Loaded plugins: product-id, refresh-packagekit
============================== repo: InstallMedia ==============================
[InstallMedia]
...

再列一下repository:
[root@mhvtlserver yum.repos.d]# yum  repolist  all
Loaded plugins: product-id, refresh-packagekit, subscription-manager
Updating Red Hat repositories.
repo id          repo name                                        status
InstallMedia     Red Hat Enterprise Linux 6.1                     enabled: 3,480
rhel-source      Red Hat Enterprise Linux 6Server - x86_64 - Sour     disabled
rhel-source-beta Red Hat Enterprise Linux 6Server Beta - x86_64 -     disabled
repolist: 3,480

InstallMedia这个repository变成了enabled,可以使用了。

测试安装:

看需要的软件libXm.so.3(这个是部分名字即可,不一定是全名)是由那个可以安装的模块提供的:

[root@mhvtlserver ~]# yum  whatprovides  libXm.so.3
Loaded plugins: product-id, refresh-packagekit, subscription-manager
Updating Red Hat repositories.
openmotif22-2.2.3-19.el6.i686 : Open Motif runtime libraries and executables
Repo        : InstallMedia
Matched from:
Other       : libXm.so.3

安装openmotif22-2.2.3-19.el6.i686:
[root@mhvtlserver ~]# yum install openmotif22-2.2.3-19.el6.i686
  …

  • 3. 安装配置和测试的安装配置:

    首先使用yum 安装一些工具软件:

    yum install zlib-devel
    yum install lsscsi
    yum install mt-st
    yum install mtx
    yum install lzo

    还有其他包,可能需要从网上下载:
    lzo-devel-2.03-3.1.el6.x86_64
    lzo-minilzo-2.03-3.1.el6.x86_64

    安装完毕后,从MHVTL网站下载所需的包:
    mhvtl-2012-08-08.tgz
    首先添加VTL用户:
    #useradd vtl
    #passwd vtl
    #mkdir  /opt/mhvtl (这个目录是存放虚拟磁带的海量存储空间)
    #chown –Rf  vtl:vtl  /opt/mhvtl

    创建mhvtl的配置目录:
    mkdir  /etc/mhvtl
    chown  -Rf vtl:vtl  /etc/mhvtl(这个目录是存放mhvtl的存储空间)

    MHVTL INSTALL:

    # tar xvf mhvtl-2012-08-08.tgz
    #cd mhvtl-1.4/
    # make distclean
    首先安装kernel部分:
    # cd kernel/
    # make distclean
    #make
    # make install
    然后其他部分:
    # cd ..
    #make
    #make install

    安装完成后,/etc/init.d/mhvtl start即可启动mhvtl, 如果没有配置文件,会自动生成缺省的配置文件。

    停止mhvtl:
    #/etc/init.d/mhvtl stop

    停止mhvtl后,为了便于其他的备份软件使用,编辑修改mhvtl配置文件,主要包括device.conf,mhvtl.conf, library_contents.xx ,如下:

    [root@mhvtlserver mhvtl]# more  /etc/mhvtl/device.conf

    VERSION: 5

    # VPD page format:
    # ...
    # NAA format is an 8 hex byte value seperated by ':'
    # Note: NAA is part of inquiry VPD 0x83
    #
    # Each 'record' is separated by one (or more) blank lines.
    # Each 'record' starts at column 1
    # Serial num max len is 10.
    # Compression: factor X enabled 0|1
    #     Where X is zlib compression factor        1 = Fastest compression
    #                                               9 = Best compression
    #     enabled 0 == off, 1 == on
    #
    # fifo: /var/tmp/mhvtl
    # If enabled, data must be read from fifo, otherwise daemon will block
    # trying to write.
    # e.g. cat /var/tmp/mhvtl (in another terminal)

    Library: 10 CHANNEL: 00 TARGET: 00 LUN: 00     (定义带库,这一行中的10是带库的标识,带库的配置则在相应的library_contents.10文件中指定)
    Vendor identification: IBM
    Product identification: 03584L32
    Product revision level: 4.02
    Unit serial number: 9890400
    NAA: 50:22:33:44:ab:00:00:00
    Home directory: /opt/mhvtl
    # fifo: /var/tmp/mhvtl

    Drive: 11 CHANNEL: 00 TARGET: 01 LUN: 00   (以下定义各个驱动器)
    Library ID: 10 Slot: 01
    Vendor identification: IBM
    Product identification: ULTRIUM-TD3
    Product revision level: 51TB
    Unit serial number: 9825111
    NAA: 10:22:33:44:ab:00:01:00
    Compression: factor 1 enabled 1
    Compression type: lzo
    Backoff: 1000
    # fifo: /var/tmp/mhvtl

    Drive: 12 CHANNEL: 00 TARGET: 02 LUN: 00
    Library ID: 10 Slot: 02
    Vendor identification: IBM
    Product identification: ULTRIUM-TD3
    Unit serial number: 9825112
    Product revision level: 51TB
    NAA: 10:22:33:44:ab:00:02:00
    Compression: factor 1 enabled 1
    Compression type: lzo
    Backoff: 1000
    # fifo: /var/tmp/mhvtl

    Drive: 13 CHANNEL: 00 TARGET: 03 LUN: 00
    Library ID: 10 Slot: 03
    Vendor identification: IBM
    Product identification: ULTRIUM-TD3
    Unit serial number: 9825113
    Product revision level: 51TB
    NAA: 10:22:33:44:ab:00:03:00
    Compression: factor 1 enabled 1
    Compression type: lzo
    Backoff: 1000
    # fifo: /var/tmp/mhvtl

    Drive: 14 CHANNEL: 00 TARGET: 04 LUN: 00
    Library ID: 10 Slot: 04
    Vendor identification: IBM
    Product identification: ULTRIUM-TD3
    Unit serial number: 9825114
    Product revision level: 51TB
    NAA: 10:22:33:44:ab:00:04:00
    Compression: factor 1 enabled 1
    Compression type: lzo
    Backoff: 1000
    # fifo: /var/tmp/mhvtl

    Library_contents.10中指定磁带,驱动器等的数量,这里磁带类型用L3---因为驱动器是ULTRIUM-TD3:
    [root@mhvtlserver mhvtl]# more library_contents.10

    Drive 1:
    Drive 2:
    Drive 3:
    Drive 4:

    Picker 1:

    MAP 1:
    MAP 2:
    MAP 3:
    MAP 4:

    # Slot 1 - ?, no gaps
    # Slot N: [barcode]
    # [barcode]
    # a barcode is comprised of three fields: [Leading] [identifier] [Trailing]
    # Leading "CLN" -- cleaning tape
    # Leading "W" -- WORM tape
    # Leading "NOBAR" -- will appear to have no barcode
    # If the barcode is at least 8 character long, then the last two characters are Trailing
    # Trailing "S3" - SDLT600
    # Trailing "X4" - AIT-4
    # Trailing "L1" - LTO 1, "L2" - LTO 2, "L3" - LTO 3, "L4" - LTO 4, "L5" - LTO 5
    # Training "LT" - LTO 3 WORM, "LU" LTO 4 WORM, "LV" LTO 5 WORM
    # Trailing "TA" - T10000+
    # Trailing "JA" - 3592+
    # Trailing "JB" - 3592E05+
    # Trailing "JW" - WORM 3592+
    # Trailing "JX" - WORM 3592E05+
    #
    Slot 1: E01001L3
    Slot 2: E01002L3
    Slot 3: E01003L3
    Slot 4: E01004L3
    Slot 5: E01005L3
    Slot 6: E01006L3
    Slot 7: E01007L3
    Slot 8: E01008L3
    Slot 9: E01009L3
    Slot 10: E01010L3
    Slot 11: E01011L3
    Slot 12: E01012L3
    Slot 13: E01013L3
    Slot 14: E01014L3
    Slot 15: E01015L3
    Slot 16: E01016L3
    Slot 17: E01017L3
    Slot 18: E01018L3
    Slot 19: E01019L3
    Slot 20: E01020L3
    Slot 21:
    Slot 22:
    Slot 23:
    Slot 24:
    Slot 25:
    Slot 26:
    Slot 27:
    Slot 28:
    Slot 29:
    Slot 30: E01030L3
    Slot 31: E01031L3
    Slot 32: E01032L3
    Slot 33: E01033L3
    Slot 34: E01034L3
    Slot 35: E01035L3
    Slot 36: E01036L3
    Slot 37: E01037L3
    Slot 38: E01038L3
    Slot 39: E01039L3

    mhvtl.conf配置文件:
    [root@mhvtlserver mhvtl]# more  /etc/mhvtl/mhvtl.conf

    # Home directory for config file(s)
    MHVTL_CONFIG_PATH=/etc/mhvtl

    # Default media capacity (500 M)
    CAPACITY=400000   (每盘磁带400G)

    # Set default verbosity [0|1|2|3]
    VERBOSE=1

    # Set kernel module debuging [0|1]
    VTL_DEBUG=0

    修改完毕mhvtl的配置文件后,启动mhvtl:
    #/etc/init.d/mhvtl start

    将其添加到chkconfig中:
    #chkconfig  --add  mhvtl
    #chkconfig mhvtl   {on|off}  (设置自动启动,这里设置为off,然后在/etc/rc.local中启动)

    查看生成的mhvtl设备:

    #lsscsi  -g

    [6:0:0:0]    mediumx IBM      03584L32         4.02  -         /dev/sg21
    [6:0:1:0]    tape    IBM      ULTRIUM-TD3      51TB  -         /dev/sg17
    [6:0:2:0]    tape    IBM      ULTRIUM-TD3      51TB  -         /dev/sg18
    [6:0:3:0]    tape    IBM      ULTRIUM-TD3      51TB  -         /dev/sg19
    [6:0:4:0]    tape    IBM      ULTRIUM-TD3      51TB  -         /dev/sg20

    B.设备测试:

    [root@mhvtlserver mhvtl]# mtx  -f  /dev/sg21  status  (/dev/sg21为03584L32 mediumx)
      Storage Changer /dev/sg21:4 Drives, 43 Slots ( 4 Import/Export )
    Data Transfer Element 0:Empty
    Data Transfer Element 1:Empty
    Data Transfer Element 2:Empty
    Data Transfer Element 3:Empty
          Storage Element 1:Full :VolumeTag=E01001L3                           
          Storage Element 2:Full :VolumeTag=E01002L3                           
          Storage Element 3:Full :VolumeTag=E01003L3                           
          Storage Element 4:Full :VolumeTag=E01004L3                           
          Storage Element 5:Full :VolumeTag=E01005L3                           
          Storage Element 6:Full :VolumeTag=E01006L3                           
          Storage Element 7:Full :VolumeTag=E01007L3                           
          Storage Element 8:Full :VolumeTag=E01008L3                           
          Storage Element 9:Full :VolumeTag=E01009L3                           
          Storage Element 10:Full :VolumeTag=E01010L3                           
          Storage Element 11:Full :VolumeTag=E01011L3                           
          Storage Element 12:Full :VolumeTag=E01012L3                           
          Storage Element 13:Full :VolumeTag=E01013L3                           
          Storage Element 14:Full :VolumeTag=E01014L3                           
          Storage Element 15:Full :VolumeTag=E01015L3                           
          Storage Element 16:Full :VolumeTag=E01016L3                           
          Storage Element 17:Full :VolumeTag=E01017L3                           
          Storage Element 18:Full :VolumeTag=E01018L3                           
          Storage Element 19:Full :VolumeTag=E01019L3                           
          Storage Element 20:Full :VolumeTag=E01020L3                           
          Storage Element 21:Empty
          Storage Element 22:Empty
          Storage Element 23:Empty
          Storage Element 24:Empty
          Storage Element 25:Empty
          Storage Element 26:Empty
          Storage Element 27:Empty
          Storage Element 28:Empty
          Storage Element 29:Empty
          Storage Element 30:Full :VolumeTag=E01030L3                           
          Storage Element 31:Full :VolumeTag=E01031L3                           
          Storage Element 32:Full :VolumeTag=E01032L3                           
          Storage Element 33:Full :VolumeTag=E01033L3                           
          Storage Element 34:Full :VolumeTag=E01034L3                           
          Storage Element 35:Full :VolumeTag=E01035L3                           
          Storage Element 36:Full :VolumeTag=E01036L3                           
          Storage Element 37:Full :VolumeTag=E01037L3                           
          Storage Element 38:Full :VolumeTag=E01038L3                           
          Storage Element 39:Full :VolumeTag=E01039L3                           
          Storage Element 40 IMPORT/EXPORT:Empty
          Storage Element 41 IMPORT/EXPORT:Empty
          Storage Element 42 IMPORT/EXPORT:Empty
          Storage Element 43 IMPORT/EXPORT:Empty

    将磁带从1号槽位装入driver 0:
    [root@mhvtlserver mhvtl]# mtx -f /dev/sg21 load 1 0
    Loading media from Storage Element 1 into drive 0...done
    [root@mhvtlserver mhvtl]# mtx -f /dev/sg21 status
      Storage Changer /dev/sg21:4 Drives, 43 Slots ( 4 Import/Export )
    Data Transfer Element 0:Full (Storage Element 1 Loaded):VolumeTag = E01001L3                           
    Data Transfer Element 1:Empty
    Data Transfer Element 2:Empty
    Data Transfer Element 3:Empty
          Storage Element 1:Empty
          Storage Element 2:Full :VolumeTag=E01002L3

    将磁带从drive0卸到1号槽位:
    [root@mhvtlserver mhvtl]# mtx -f /dev/sg21 unload 1 0
    Unloading drive 0 into Storage Element 1...done
    [root@mhvtlserver mhvtl]# mtx -f /dev/sg21 status
      Storage Changer /dev/sg21:4 Drives, 43 Slots ( 4 Import/Export )
    Data Transfer Element 0:Empty
    Data Transfer Element 1:Empty
    Data Transfer Element 2:Empty
    Data Transfer Element 3:Empty
          Storage Element 1:Full :VolumeTag=E01001L3

    读写测试:
    载入39号槽位的磁带到drive 0 (/dev/IBMtape0)并读写:
    [root@mhvtlserver mhvtl]# mtx -f /dev/sg21 load 39 0
    Loading media from Storage Element 39 into drive 0...done
    [root@mhvtlserver mhvtl]# cd /opt/mhvtl/E01039L3/
    [root@mhvtlserver E01039L3]# ls -l
    total 4
    -rw-rw---- 1 vtl vtl    0 Aug 14 15:43 data
    -rw-rw---- 1 vtl vtl    0 Aug 14 15:43 indx
    -rw-rw---- 1 vtl vtl 1536 Aug 29 17:55 meta

    查看
    [root@mhvtlserver E01039L3]# mt  -f  /dev/IBMtape0  status
    SCSI 2 tape drive:
    File number=-1, block number=0, partition=0.
    Tape block size 0 bytes. Density code 0x44 (LTO-3).
    Soft error count since last status=0
    General status bits on (41000000):
    BOT ONLINE

    查看没有磁带的驱动器:
    [root@mhvtlserver E01039L3]# mt -f /dev/IBMtape1 status
    SCSI 2 tape drive:
    File number=-1, block number=-1, partition=0.
    Tape block size 0 bytes. Density code 0x44 (LTO-3).
    Soft error count since last status=0
    General status bits on (40000):
    DR_OPEN

    写入磁带:
    [root@mhvtlserver E01039L3]# tar  cvvf  /dev/IBMtape0 /tmp/software/RHEL6.1X86_64.iso
    tar: Removing leading `/' from member names
    -rw-r--r-- root/root 3600842752 2012-08-16 16:33 /tmp/software/RHEL6.1X86_64.iso
    [root@mhvtlserver E01039L3]# ls -l
    total 3617140
    -rw-rw---- 1 vtl vtl 3520277200 Aug 29 18:20 data
    -rw-rw---- 1 vtl vtl  180043264 Aug 29 18:20 indx
    -rw-rw---- 1 vtl vtl       1544 Aug 29 18:20 meta

    读:(列出磁带内容)
    [root@mhvtlserver E01039L3]# mt -f /dev/IBMtape0 rewind
    [root@mhvtlserver E01039L3]# tar tvf /dev/IBMtape0
    -rw-r--r-- root/root 3600842752 2012-08-16 16:33 tmp/software/RHEL6.1X86_64.iso

    擦掉磁带内容再查看:
    [root@mhvtlserver E01039L3]# mt  -f  /dev/IBMtape0  erase
    [root@mhvtlserver E01039L3]# tar  tvf /dev/IBMtape0
    tar: /dev/IBMtape0: Cannot read: Input/output error
    tar: At beginning of tape, quitting now
    tar: Error is not recoverable: exiting now
    [root@mhvtlserver E01039L3]# ls -l
    total 4
    -rw-rw---- 1 vtl vtl    0 Aug 29 18:23 data
    -rw-rw---- 1 vtl vtl    0 Aug 29 18:23 indx
    -rw-rw---- 1 vtl vtl 1536 Aug 29 18:23 meta
    [root@mhvtlserver E01039L3]# mt -f /dev/IBMtape0  rewind
    [root@mhvtlserver E01039L3]# mt -f /dev/IBMtape0  status
    SCSI 2 tape drive:
    File number=-1, block number=0, partition=0.
    Tape block size 0 bytes. Density code 0x44 (LTO-3).
    Soft error count since last status=0
    General status bits on (41000000):
    BOT ONLINE

    将该磁带放回39号槽位:
    [root@mhvtlserver E01039L3]# mtx -f  /dev/sg21  unload  39  0
    Unloading drive 0 into Storage Element 39...done
    [root@mhvtlserver E01039L3]# mtx  -f  /dev/sg21  status
      Storage Changer /dev/sg21:4 Drives, 43 Slots ( 4 Import/Export )
    Data Transfer Element 0:Empty
    Data Transfer Element 1:Empty
    Data Transfer Element 2:Empty
    Data Transfer Element 3:Empty

          Storage Element 39:Full :VolumeTag=E01039L3

     

  • 4. 和Qlogic Target Mode driver安装配置

本机上的MHVTL配置完成并测试通过后,需要配置SCST和Qlogic HBA卡的target mode driver。

 

安装openssl ,openssl-devel等软件:

#yum install openssl openssl-devel  gcc  ncurses-devel  kernel-devel  lsscsi patch subversion

从Qlogic网站或是下列地址:

下载最新Firmware,并保存到/lib/firmware,否则在使用Qlogic HBA卡时可能会报错:

qla2xxx 0000:13:00.0: Firmware image unavailable.

qla2xxx 0000:13:00.0: Failed to initialize adapter

本例中使用的是:

ql2400_fw.bin

从 下载scst,  scstadmin, iscsi-scst, qla2x00t的源代码:

iscsi-scst-2.2.0.tar

qla2x00t-2.2.0.tar

scst-2.2.0.tar

scstadmin-2.2.0.tar

并使用

#tar  xvf  xxxx.tar 解压缩各个tar包。

如果没有mockbuild用户那么使用useradd 添加mockbuild用户,用来编译安装内核代码。

移去当前正在使用的由系统提供的qla2xxx模块,并在下次启动时屏蔽:

#rmmod qla2xxx

#echo  blacklist  qla2xxx >/etc/modprobe.d/blacklist-qla2xxx.conf

#reboot

安装Kernel Source:
KERNEL BUILD:
[root@mhvtlserver ~]# cd  ~
[root@mhvtlserver ~]# rm -fr rpmbuild/
[root@mhvtlserver ~]# cd Downloads/
[root@mhvtlserver Downloads]# ls
..
kernel-2.6.32-131.0.15.el6.src.rpm
qla2x00t-2.2.0.tar
..
[root@mhvtlserver Downloads]# rpm -ivh kernel-2.6.32-131.0.15.el6.src.rpm
warning: kernel-2.6.32-131.0.15.el6.src.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
   1:kernel                 ########################################### [100%]

内核源代码安装完成后,开始做一些修改,然后再重新编译内核:

[root@mhvtlserver Downloads]# cd  /root
[root@mhvtlserver ~]# cd rpmbuild/    (RHEL6.1的rpmbuild放在这个目录下)
[root@mhvtlserver rpmbuild]# ls
SOURCES  SPECS
[root@mhvtlserver rpmbuild]# cd SPECS/
[root@mhvtlserver SPECS]# ls
kernel.spec
[root@mhvtlserver SPECS]# rngd -r /dev/urandom (生成随机数,否则下面的rpmbuild的过程会很漫长)

[root@mhvtlserver rpmbuild]# mkdir -p BUILD
[root@mhvtlserver rpmbuild]# chown mockbuild BUILD

执行kernel.spec的"%prep"部分,主要是解开源代码并打相应补丁:
[root@mhvtlserver SPECS]# rpmbuild -bp kernel.spec

到包含qla2xxx驱动模块的目录下:
[root@mhvtlserver rpmbuild]# cd  /root/rpmbuild/BUILD
[root@mhvtlserver BUILD]# cd kernel-2.6.32-131.0.15.el6/
[root@mhvtlserver kernel-2.6.32-131.0.15.el6]# cd linux-2.6.32-131.0.15.el6.x86_64/
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]# pwd
/root/rpmbuild/BUILD/kernel-2.6.32-131.0.15.el6/linux-2.6.32-131.0.15.el6.x86_64
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]# cd drivers/scsi/

将原有的kernel source自带的qla2xxx目录改名保存:
[root@mhvtlserver scsi]# mv qla2xxx/ qla2xxx_orig

将解压缩的qla2x00t-2.2.0改名,并建立链接,也就是用下载的qla2x00t-2.2.0中的qla2xxx驱动替代原有的kenel source自带的qla2xxx驱动:
[root@mhvtlserver scsi]#mv  /root/Downloads/qla2x00t-2.2.0  /root/Downloads/qla2x00t
[root@mhvtlserver scsi]# ln  -s  /root/Downloads/qla2x00t/   qla2xxx

然后对现有的kernel source打补丁,包括两个补丁,这两个补丁来分别自两个不同的包,它们是scst-2.2.0.tar和iscsi-scst-2.2.0.tar:

[root@mhvtlserver scsi]# cd  ../..
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]# pwd
/root/rpmbuild/BUILD/kernel-2.6.32-131.0.15.el6/linux-2.6.32-131.0.15.el6.x86_64
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]# patch -p1 < /root/Downloads/scst-2.2.0/kernel/rhel/scst_exec_req_fifo-2.6.32.patch
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]# patch -p1 < /root/Downloads/iscsi-scst-2.2.0/kernel/patches/rhel/put_page_callback-2.6.32-131.patch

使用GUI配置内核,这个比字符方式要简单多了:
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]# make xconfig
选择菜单:
"Device Drivers -> SCSI device support -> SCSI low-level drivers"
确保”QLogic QLA2XXX Fibre Channel Support -> QLogic 2xxx target mode support”
被选中。然后,移动到"SCSI debugging host simulator" 并选中,
保存退出。

修改内核Makefile,加上内核扩展名:
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]#vi Makefile
EXTRAVERSION = -scst

开始编译内核:
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]# make
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]# make modules; mkdir -p /lib/modules/2.6.18-scst
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]#make modules_install && make install

编译完毕后重启,选择新内核启动:
[root@mhvtlserver linux-2.6.32-131.0.15.el6.x86_64]#reboot (select the '-scst' kernel.)

这样,在编译内核的过程中,新的QLogic 的 driver qla2xxx就顺带被编译并加入到了新内核中,新内核中的Qlogic HBA卡的驱动此时是支持initiator/target  mode的,这个可以使用
#modinfo qla2xxx
查看,其中有qlini_mode参数,允许选择target mode.

[root@mhvtlserver scst-2.2.0]# cd /root/Downloads/scst-2.2.0

生成SCST Release版本,只有很少的告警信息,没有调试信息,然后安装:
[root@mhvtlserver scst-2.2.0]# make  debug2release
[root@mhvtlserver scst-2.2.0]# cd  /root/Downloads/scst-2.2.0/src

[root@mhvtlserver src]# vi  Makefile
Uncomment: EXTRA_CFLAGS += -DCONFIG_SCST_STRICT_SERIALIZING
  (Needed for tape emulation)
[root@mhvtlserver scst-2.2.0]# cd /root/Downloads/scst-2.2.0
[root@mhvtlserver scst-2.2.0]# make
[root@mhvtlserver scst-2.2.0]# make install

之后,在/lib/modules/2.6.32-scst/extra下会有:
..
-rw-r--r-- 1 root root 3734064 Aug 20 12:28 scst.ko
..
[root@mhvtlserver dev_handlers]# pwd
/lib/modules/2.6.32-scst/extra/dev_handlers
[root@mhvtlserver dev_handlers]# ls -l
total 3380
-rw-r--r-- 1 root root 306372 Aug 20 12:28 scst_cdrom.ko
-rw-r--r-- 1 root root 288539 Aug 20 12:28 scst_changer.ko
-rw-r--r-- 1 root root 344607 Aug 20 12:28 scst_disk.ko
-rw-r--r-- 1 root root 311477 Aug 20 12:28 scst_modisk.ko
-rw-r--r-- 1 root root 288563 Aug 20 12:28 scst_processor.ko
-rw-r--r-- 1 root root 288499 Aug 20 12:28 scst_raid.ko
-rw-r--r-- 1 root root 313697 Aug 20 12:28 scst_tape.ko
-rw-r--r-- 1 root root 624139 Aug 20 12:28 scst_user.ko
-rw-r--r-- 1 root root 672542 Aug 20 12:28 scst_vdisk.ko

说明:
scst - SCST itself.
scst_disk - device handler for disks (type 0).
scst_tape - device handler for tapes (type 1).
scst_processor - device handler for processors (type 3).
scst_cdrom - device handler for CDROMs (type 5).
scst_modisk - device handler for MO disks (type 7).
scst_changer - device handler for medium changers (type 8).
scst_raid - device handler for storage array controller (e.g. raid) (type C).
scst_vdisk - device handler for virtual disks (file, device or ISO CD image)..
scst_user - user space device handler.

在/usr/local/include/scst/下也会有一些文件。

下一步,生成Qlogic target mode driver add-on,并安装:
[root@mhvtlserver scst-2.2.0]#cd /root/Downloads/qla2x00t/qla2x00-target

修改SCST  include和src文件的路径,否则会报错:
[root@mhvtlserver src]# vi Makefile

#SCST_INC_DIR := /usr/local/include/scst
#SCST_DIR := $(SCST_INC_DIR)
SCST_INC_DIR := /root/Downloads/scst-2.2.0/include
SCST_DIR := /root/Downloads/scst-2.2.0/src

[root@mhvtlserver qla2x00-target]# make clean
[root@mhvtlserver qla2x00-target]# make
[root@mhvtlserver qla2x00-target]# make uninstall
[root@mhvtlserver qla2x00-target]# make install

编译完成后,在/lib/modules/2.6.32-scst/extra下生成:
-rw-r--r-- 1 root root  982816 Aug 20 12:38 qla2x00tgt.ko

 

手动加载这些模块:
#modprobe scst
#modprobe scst_changer
#modprobe scst_tape

应该会生成/sys/kernel/scst_tgt目录:
[root@mhvtlserver scst_tgt]# ls -F
device_groups/  handlers/            max_tasklet_cmd  sgv/      threads      version
devices/        last_sysfs_mgmt_res  setup_id         targets/  trace_level

handlers目录下应该有:
[root@mhvtlserver scst_tgt]# ls -F handlers/
dev_changer/  dev_tape/ dev_tape_perf/        ...
dmesg看应该有:
Aug  6 13:20:44 mhvtlserver kernel: [2885]: scst: init_scst:2362:SCST version 2.2.0 loaded successfully (max mem for commands 1964MB, per device 785MB)
Aug  6 13:20:44 mhvtlserver kernel: [2916]: scst: __scst_register_dev_driver:1406:Device handler "dev_tape" for type 1 registered successfully
Aug  6 13:20:44 mhvtlserver kernel: [2916]: scst: __scst_register_dev_driver:1406:Device handler "dev_tape_perf" for type 1 registered successfully
Aug  6 13:20:44 mhvtlserver kernel: [2923]: scst: __scst_register_dev_driver:1406:Device handler "dev_changer" for type 8 registered successfully

然后加载qla2xxx和qla2x00tgt:

首先修改modprobe配置,去掉blacklist-qla2xxx.conf,然后添加qla.conf:
[root@mhvtlserver modprobe.d]# more  /etc/modprobe.d/qla.conf
options qla2xxx  qlini_mode="disabled"

这里qlini_mode说明如下:
When qla2xxx compiled with CONFIG_SCSI_QLA2XXX_TARGET enabled, it has
parameter "qlini_mode", which determines when initiator mode will be
enabled. Possible values:
- "exclusive" (default) - initiator mode will be enabled on load,
disabled on enabling target mode and then on disabling target mode
enabled back.
- "disabled" - initiator mode will never be enabled.
- "enabled" - initiator mode will always stay enabled.

加载这些模块:
#modprobe qla2xxx
#modprobe qla2x00tgt

dmesg:
这个是qla2xxx:
Aug  6 13:20:32 mhvtlserver kernel: QLogic Fibre Channel HBA Driver: 8.02.01-k4-tgt
Aug  6 13:20:32 mhvtlserver kernel: qla2xxx 0000:05:00.0: PCI INT A -> GSI 35 (level, low) -> IRQ 35
Aug  6 13:20:32 mhvtlserver kernel: qla2xxx 0000:05:00.0: Found an ISP2432, irq 35, iobase 0xffffc90011836000
Aug  6 13:20:32 mhvtlserver kernel: qla2xxx 0000:05:00.0: Configuring PCI space...
Aug  6 13:20:32 mhvtlserver kernel: qla2xxx 0000:05:00.0: Configure NVRAM parameters...
Aug  6 13:20:32 mhvtlserver kernel: qla2xxx 0000:05:00.0: Verifying loaded RISC code...
Aug  6 13:20:32 mhvtlserver kernel: qla2xxx 0000:05:00.0: firmware: requesting ql2400_fw.bin
Aug  6 13:20:32 mhvtlserver kernel: scsi5 : qla2xxx
Aug  6 13:20:32 mhvtlserver kernel: qla2xxx 0000:05:00.0:
Aug  6 13:20:32 mhvtlserver kernel: QLogic Fibre Channel HBA Driver: 8.02.01-k4-tgt

qla2x00tgt:
Aug 20 12:34:03 mhvtlserver kernel: [17037]: q2t_init:6546:qla2x00t: Initializing QLogic Fibre Channel HBA Driver target mode addon version 2.2.0
Aug 20 12:34:03 mhvtlserver kernel: scst: Target 21:00:00:24:ff:05:24:b4 for template qla2x00t registered successfully
Aug 20 12:34:03 mhvtlserver kernel: [17037]: q2t_add_target:5758:qla2x00t(0): using 64 Bit PCI addressing
Aug 20 12:34:03 mhvtlserver kernel: [17037]: q2t_target_detect:602:qla2x00t: Target mode driver for QLogic 2x00 controller registered successfully
Aug 20 12:34:03 mhvtlserver kernel: scst: Target template qla2x00t registered successfully

此时,应该可以在其他的主机上能够看到这些被输出的设备。(客户机可能需要重新启动,或是重新扫描硬件)

在另外一台机器上:
#dmesg

scsi 2:0:2:0: Medium Changer    IBM      03584L32         4.02 PQ: 0 ANSI: 5
ch0: type #1 (mt): 0x100+1 [medium transport]
ch0: type #2 (st): 0x400+39 [storage]
ch0: type #3 (ie): 0x200+4 [import/export]
ch0: type #4 (dt): 0x1+4 [data transfer]
ch0: dt 0x1: ID/LUN unknown
ch0: dt 0x2: ID/LUN unknown
ch0: dt 0x3: ID/LUN unknown
ch0: dt 0x4: ID/LUN unknown
ch0: INITIALIZE ELEMENT STATUS, may take some time ...
ch0: ... finished
ch 2:0:2:0: Attached scsi changer ch0
ch 2:0:2:0: Attached scsi generic sg13 type 8
lin_tape: detected device at host 2, channel 0, id 2, lun 0
IBM 03584L32    4.02
lin_tape: attached IBMchanger0
scsi 2:0:2:1: Sequential-Access IBM      ULTRIUM-TD3      51TB PQ: 0 ANSI: 5
lin_tape 2:0:2:1: Attached scsi generic sg14 type 1
lin_tape: detected device at host 2, channel 0, id 2, lun 1
IBM ULTRIUM-TD3 51TB
lin_tape: attached IBMtape0
scsi 2:0:2:2: Sequential-Access IBM      ULTRIUM-TD3      51TB PQ: 0 ANSI: 5
lin_tape 2:0:2:2: Attached scsi generic sg15 type 1
lin_tape: detected device at host 2, channel 0, id 2, lun 2
IBM ULTRIUM-TD3 51TB
lin_tape: attached IBMtape1
scsi 2:0:2:3: Sequential-Access IBM      ULTRIUM-TD3      51TB PQ: 0 ANSI: 5
lin_tape 2:0:2:3: Attached scsi generic sg16 type 1
lin_tape: detected device at host 2, channel 0, id 2, lun 3
IBM ULTRIUM-TD3 51TB
lin_tape: attached IBMtape2
scsi 2:0:2:4: Sequential-Access IBM      ULTRIUM-TD3      51TB PQ: 0 ANSI: 5
lin_tape 2:0:2:4: Attached scsi generic sg17 type 1
lin_tape: detected device at host 2, channel 0, id 2, lun 4
IBM ULTRIUM-TD3 51TB
lin_tape: attached IBMtape3

修改/etc/init.d/scst文件,自动加载scst scst_changer scst_tape qla2x00tgt等驱动模块:
找到:
parse_scst_conf() {
SCST_MODULES=…
修改为:
parse_scst_conf() {
SCST_MODULES="scst scst_tape scst_changer scst_disk scst_vdisk qla2x00tgt" (scst_disk,scst_vdisk不用的话可以去掉)

通过lsscsi的结果创建/etc/scst.conf文件,可以用脚本完成:
[root@mhvtlserver init.d]# more /usr/local/bin/crscstconf.sh
# Build /etc/scst.conf
# Header of scst.conf
echo "# Header Info" >/tmp/scst.tmp
echo "setup_id 0x1234" >>/tmp/scst.tmp
echo "# Changer Devices " >>/tmp/scst.tmp

# define dev_changer

echo "HANDLER dev_changer {" >>/tmp/scst.tmp
lsscsi -g| grep mediumx | awk '{print $1}'| cut -d "[" -f2| cut -d "]" -f1| while read each1; do
echo "  DEVICE $each1" >>/tmp/scst.tmp
done
echo "}" >>/tmp/scst.tmp

# define dev_tape

echo "# Tape device" >>/tmp/scst.tmp
echo "HANDLER dev_tape {" >>/tmp/scst.tmp
lsscsi -g| grep tape | awk '{print $1}'| cut -d "[" -f2| cut -d "]" -f1 | while read each2; do
echo "  DEVICE $each2" >>/tmp/scst.tmp
done
echo "}" >>/tmp/scst.tmp

# define Qlogic part.
echo "# Qlogic Devices " >>/tmp/scst.tmp
echo "TARGET_DRIVER qla2x00t {" >>/tmp/scst.tmp
echo "    TARGET 21:00:00:24:ff:05:24:b4 {" >>/tmp/scst.tmp

int=0

lsscsi -g | grep -e tape -e mediumx|awk '{print $1}'| cut -d "[" -f2| cut -d "]" -f1| while read each3; do
echo "          LUN $int $each3" >>/tmp/scst.tmp
int=`expr $int + 1`
done
echo "            enabled 1" >>/tmp/scst.tmp
echo "  }" >>/tmp/scst.tmp
echo "}" >>/tmp/scst.tmp

if [ -f /etc/scst.conf ]; then
cp -f /etc/scst.conf /etc/scst.conf_`date +%m%d%y%H%M%S`
echo Created backup of  existing /etc/scst.conf as /etc/scst.conf_`date +%m%d%y%H%M%S`
fi
cat /tmp/scst.tmp >/etc/scst.conf
echo Created new /etc/scst.conf
rm -f /tmp/scst.tmp

echo --- Done ---

这个脚本中的TARGET 21:00:00:24:ff:05:24:b4等需要从/sys/kernel/scst_tgt/targets目录下得到:
[root@mhvtlserver targets]# pwd
/sys/kernel/scst_tgt/targets
[root@mhvtlserver targets]# ls
qla2x00t
[root@mhvtlserver targets]# cd qla2x00t/
[root@mhvtlserver qla2x00t]# ls
21:00:00:24:ff:05:24:b4  mgmt  trace_level  version
[root@mhvtlserver qla2x00t]#

执行这个脚本,生成:
[root@mhvtlserver qla2x00t]# more /etc/scst.conf
# Header Info
setup_id 0x1234
# Changer Devices
HANDLER dev_changer {
        DEVICE 6:0:0:0
}
# Tape device
HANDLER dev_tape {
        DEVICE 6:0:1:0
        DEVICE 6:0:2:0
        DEVICE 6:0:3:0
        DEVICE 6:0:4:0
}
# Qlogic Devices
TARGET_DRIVER qla2x00t {
    TARGET 21:00:00:24:ff:05:24:b4 {
                LUN 0 6:0:0:0
                LUN 1 6:0:1:0
                LUN 2 6:0:2:0
                LUN 3 6:0:3:0
                LUN 4 6:0:4:0
            enabled 1
        }
}

然后,启动scst, 其他的机器就只能看到这个/etc/scst.conf输出的设备:
#service scst stop
#service scst start

修改/etc/rc.local,改为自动启动:
[root@mhvtlserver qla2x00t]# more /etc/rc.local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local
# First, create the config file /etc/scst.conf for scst.
/usr/local/bin/crscstconf.sh
# Then start scst.
service scst start

这样系统启动完毕后,会自动加载qla2xxx(由内核加载), scst scst_changer scst_tape qla2x00tgt等模块,并根据刚刚生成的/etc/scst.conf来配置那些需要输出的设备。

阅读(16149) | 评论(0) | 转发(1) |
给主人留下些什么吧!~~