多少人爱你青春欢畅的时辰, 爱你的美丽假意和真心, 只有一个人爱你那朝圣者的灵魂, 爱你衰老了的连上痛苦的皱纹.
分类: 云计算
2014-07-17 12:58:11
2 配置步骤
配置参数表
控制器A
名称: UCStorage002
控制器序列号 850000XXXXXX
硬盘数量 28
RAID级别
vif名称 VIF001
vif1 端口 e0a,e0b e2a、e2b
Vif模式 LACP
VIF001的IP 地址 10.16.255.33/255.255.255.0
e0M端口IP地址 172.18.73.249/255.255.255.0
SP端口IP地址 172.18.73.247/255.255.255.0
SP网关 172.18.73.254 172.18.73.254
管理服务器地址 10.16.255.206
2.1 设置磁盘归属,创建ROOT卷
#启动控制器A
Phoenix TrustedCore(tm) Server
Copyright 1985-2006 Phoenix Technologies Ltd. All Rights Reserved BIOS version: 5.1.1
Portions Copyright (c) 2007-2011 NetApp, Inc. All Rights Reserved
CPU = 1 Processors Detected, Cores per Processor = 2 Intel(R) Xeon(R) CPU E5220 @ 2.33GHz Testing RAM
512MB RAM tested 5120MB RAM installed 6144 KB L2 Cache
System BIOS shadowed
USB 2.0: MICRON eUSB DISK
BIOS is scanning PCI Option ROMs, this will take few seconds…
...................
Boot Loader version 3.2
Copyright (C) 2000-2003 Broadcom Corporation. Portions Copyright (C) 2002-2008 NetApp
CPU Type: Intel(R) Xeon(R) CPU E5220 @ 2.33GHz
Starting AUTOBOOT press Ctrl-C to abort...
Loading X86_64/kernel/primary.krn:0x200000/49123928 0x30d9258/27512016 0x4b15f28/7980237 0x52b23f5/3 Entry at 0x00202018
Starting program at 0x00202018
MTRR0, will be reset to map region 3G to 4G as UC BBSP on this platform is 0x0000000000000180
ipmi_init_alloc_request_pool succeed (pool entries 0x80,size per entry 0xe0) ipmi_init0: DONE.
Press CTRL-C for special boot menu #进入特殊启动菜单
IPMI:IPMI device rev. 2, firmware rev. 1.0, version 2.0 IPMI:Number of channels 0 ipmi_init1: DONE.
Special boot options menu will be available. Chelsio T3 RDMA Driver - version 0.1 Ipspace "iwarp-ipspace" created
Tue Mar 20 08:49:03 GMT [fci.initialization.failed:error]: Initialization failed on Fibre Channel adapter 0c. Tue Mar 20 08:49:03 GMT [fci.initialization.failed:error]: Initialization failed on Fibre Channel adapter 0d.
NetApp Release 7.3.6: Thu Jul 7 01:48:12 PDT 2011 Copyright (c) 1992-2011 NetApp.
Starting boot on Tue Mar 20 08:48:45 GMT 2012
Tue Mar 20 08:49:04 GMT [nvram.battery.turned.on:info]: The NVRAM battery is turned ON. It is turned OFF during system shutdown.
Tue Mar 20 08:49:09 GMT [diskown.isEnabled:info]: software ownership has been enabled for this system
Tue Mar 20 08:49:13 GMT [config.noPartnerDisks:CRITICAL]: No disks were detected for the partner; this node will be unable to takeover correctly
(1) Normal boot.
(2) Boot without /etc/rc. (3) Change password.
(4) Assign ownership and initialize disks for root volume. (4a) Same as option 4, but create a flexible root volume.
-5)? 5 #进入维护模式
Tue Mar 20 08:49:16 GMT [shelf.config.mpha:info]: All attached storage on the system is multi-pathed HA.
You have selected the maintenance boot option:
the system has booted in maintenance mode allowing the following operations to be performed:
? disk fcadmin fcstat sasadmin
sasstat acpadmin halt help ifconfig
raid_config storage sesdiag sysconfig version
vol aggr sldiag environment led_on
led_off sata acorn scsi disk_list
haconfig fcdiag fctest disktest diskcopy
xortest faltest outb disk_mung
Type "help
In a cluster, you MUST ensure that the partner is (and remains) down, or that takeover is manually disabled on the partner node, because clustering software is not started or fully enabled in Maintenance mode.
FAILURE TO DO SO CAN RESULT IN YOUR FILESYSTEMS BEING DESTROYED Continue with boot? y
Tue Mar 20 08:49:40 GMT [mgr.boot.reason_ok:notice]: System rebooted after a halt command. Tue Mar 20 08:49:41 GMT [console_login_mgr:info]: root logged in from console > *> Ipspace "acp-ipspace" created
*> disk show –a #查看当前控制器下的磁盘
Local System ID: 1574659072
R POOL SERIAL NUMBER ------------ ------------- ----- -------------
0a.00.10 Not Owned NONE N000ZK5L 0a.00.2 Not Owned NONE N000J71L 0a.00.5 Not Owned NONE N000J5ML 0a.00.9 Not Owned NONE N000ZN4L 0a.00.1 Not Owned NONE N000J65L 0a.00.0 Not Owned NONE J830SRPL 0a.00.3 Not Owned NONE N000V8KL 0a.00.7 Not Owned NONE N000YG6L 0a.00.11 Not Owned NONE N000XZVL 0a.00.4 Not Owned NONE N001YL7L 0a.00.8 Not Owned NONE N002K7KL 0a.00.6 Not Owned NONE N000J0WL
*> disk assign 0a.00.0 #把磁盘0a.00.0分配给当前控制器
disk assign: DisTk assigned but uunable to obtain eowner name. Re- run 'disk assignM' with -o optiona to specify namer 20 08:53:39 GMT [diskown.changingOwner:info]: changing ownership for disk 0a.00.0 (S/N J830SRPL) from unowned (ID -1) to (ID 1574659072) .
*> disk assign 0a.00.1 #把磁盘0a.00.1分配给当前控制器
disk assign: DisTk assigned but uunable to obtain eowner name. Re- run 'disk assignM' with -o optiona to specify namer 20 08:53:57 GMT [diskown.changingOwner:info]: changing ownership for disk 0a.00.1 (S/N N000J65L) from unowned (ID -1) to (ID 1574659072) .
*> disk assign 0a.00.2 #把磁盘0a.00.2分配给当前控制器
disk assign: DisTk assigned but uunable to obtain eowner name. Re- run 'disk assignM' with -o optiona to specify namer 20 08:53:59 GMT [diskown.changingOwner:info]: changing ownership for disk 0a.00.2 (S/N N000J71L) from unowned (ID -1) to (ID 1574659072)
*> disk assign 0a.00.3 #把磁盘0a.00.3分配给当前控制器
disk assign: DisTk assigned but uunable to obtain eowner name. Re- run 'disk assignM' with -o optiona to specify namer 20 08:54:10 GMT [diskown.changingOwner:info]: changing ownership for disk 0a.00.3 (S/N N000V8KL) from unowned (ID -1) to (ID 1574659072)
Page 7
NETAPP存储系统安装、配置和维护手册
.
*> disk assign 0a.00.4 #把磁盘0a.00.4分配给当前控制器
disk assign: DisTk assigned but uunable to obtain eowner name. Re- run 'disk assignM' with -o optiona to specify namer.
20 08:54:13 GMT [diskown.changingOwner:info]: changing ownership for disk 0a.00.4 (S/N N001YL7L) from unowned (ID -1) to (ID 1574659072)
*> disk assign 0a.00.5 #把磁盘0a.00.5分配给当前控制器
disk assign: DisTk assigned but uunable to obtain eowner name. Re- run 'disk assignM' with -o optiona to specify namer.
20 08:54:19 GMT [diskown.changingOwner:info]: changing ownership for disk 0a.00.5 (S/N N000J5ML) from unowned (ID -1) to (ID 1574659072)
*> disk show –v #查看磁盘的归属关系 Local System ID: 1574659072
DISK OWNER POOL SERIAL NUMBER ------------ ------------- ----- -------------
0a.00.10 Not Owned NONE N000ZK5L 0a.00.3 (1574659072) Pool0 N000V8KL 0a.00.9 Not Owned NONE N000ZN4L 0a.00.2 (1574659072) Pool0 N000J71L 0a.00.1 (1574659072) Pool0 N000J65L 0a.00.4 (1574659072) Pool0 N001YL7L 0a.00.7 Not Owned NONE N000YG6L 0a.00.11 Not Owned NONE N000XZVL 0a.00.5 (1574659072) Pool0 N000J5ML 0a.00.8 Not Owned NONE N002K7KL 0a.00.6 Not Owned NONE N000J0WL 0a.00.0 (1574659072) Pool0 J830SRPL
*> disk show –v #拔掉多的4块硬盘后,查看磁盘的归属关系 Local System ID: 1574659072
Tue Mar 20 08:55:02 GMT [disk.readReservationFailed:error]: Disk read reservation failed on 0a.00.11 CDB 0x5e:01 - SCSI:no sense (0 0 0)
Tue Mar 20 08:55:02 GMT [diskown.errorDuringIO:error]: error 25 (no valid path to disk) on disk 0a.00.11 (S/N N000XZVL) while reading reservation state
Tue Mar 20 08:55:04 GMT [disk.readReservationFailed:error]: Disk read reservation failed on 0a.00.10 CDB 0x5e:01 - SCSI:no sense (0 0 0)
Tue Mar 20 08:55:04 GMT [diskown.errorDuringIO:error]: error 25 (no valid path to disk) on disk 0a.00.10 (S/N N000ZK5L) while reading reservation state
Tue Mar 20 08:55:05 GMT [disk.readReservationFailed:error]: Disk read reservation failed on 0a.00.9 CDB 0x5e:01 - SCSI:no sense (0 0 0)
Tue Mar 20 08:55:05 GMT [diskown.errorDuringIO:error]: error 25 (no valid path to disk) on disk 0a.00.9 (S/N N000ZN4L) while reading reservation state
Tue Mar 20 08:55:07 GMT [disk.readReservationFailed:error]: Disk read reservation failed on 0a.00.8 CDB 0x5e:01 - SCSI:no sense (0 0 0)
Tue Mar 20 08:55:07 GMT [diskown.errorDuringIO:error]: error 25 (no valid path to disk) on disk 0a.00.8 (S/N N002K7KL) while reading reservation state
DISK OWNER POOL SERIAL NUMBER ------------ ------------- ----- -------------
0a.00.3 (1574659072) Pool0 N000V8KL 0a.00.2 (1574659072) Pool0 N000J71L 0a.00.1 (1574659072) Pool0 N000J65L 0a.00.4 (1574659072) Pool0 N001YL7L 0a.00.7 Not Owned NONE N000YG6L 0a.00.5 (1574659072) Pool0 N000J5ML 0a.00.6 Not Owned NONE N000J0WL
0a.00.0 (1574659072) Pool0 J830SRPL
*> halt
Phoenix TrustedCore(tm) Server
Copyright 1985-2006 Phoenix Technologies Ltd. All Rights Reserved BIOS version: 5.1.1
Portions Copyright (c) 2007-2011 NetApp, Inc. All Rights Reserved
CPU = 1 Processors Detected, Cores per Processor = 2 Intel(R) Xeon(R) CPU E5220 @ 2.33GHz Testing RAM
512MB RAM tested 5120MB RAM installed 6144 KB L2 Cache
System BIOS shadowed
USB 2.0: MICRON eUSB DISK
BIOS is scanning PCI Option ROMs, this will take few seconds... +++++++++++++++++++
Boot Loader version 3.2
Copyright (C) 2000-2003 Broadcom Corporation. Portions Copyright (C) 2002-2008 NetApp
CPU Type: Intel(R) Xeon(R) CPU E5220 @ 2.33GHz LOADER-A
*> autoboot
Loading X86_64/kernel/primary.krn:0x200000/49123928 0x30d9258/27512016 0x4b15f28/7980237 0x52b23f5/3 Entry at 0x00202018
Starting program at 0x00202018
MTRR0, will be reset to map region 3G to 4G as UC
BBSP on this platform is 0x0000000000000180
ipmi_init_alloc_request_pool succeed (pool entries 0x80,size per entry 0xe0) ipmi_init0: DONE.
Press CTRL-C for special boot menu
IPMI:IPMI device rev. 2, firmware rev. 1.0, version 2.0 IPMI:Number of channels 0 ipmi_init1: DONE.
Chelsio T3 RDMA Driver - version 0.1 Ipspace "iwarp-ipspace" created
Tue Mar 20 08:58:55 GMT [fci.initialization.failed:error]: Initialization failed on Fibre Channel adapter 0d. Tue Mar 20 08:58:55 GMT [fci.initialization.failed:error]: Initialization failed on Fibre Channel adapter 0c.
NetApp Release 7.3.6: Thu Jul 7 01:48:12 PDT 2011 Copyright (c) 1992-2011 NetApp.
Starting boot on Tue Mar 20 08:58:37 GMT 2012
Tue Mar 20 08:58:56 GMT [nvram.battery.turned.on:info]: The NVRAM battery is turned ON. It is turned OFF during system shutdown.
Tue Mar 20 08:59:01 GMT [diskown.isEnabled:info]: software ownership has been enabled for this system Special boot options menu will be available.
(1) Normal boot.
(2) Boot without /etc/rc. (3) Change password.
(4) Initialize owned disks (6 disks are owned by this filer).
(4a) Same as option 4, but create a flexible root volume.
(5) Maintenance mode boot.
Selection (1-5)? 4a
Tue Mar 20 08:59:08 GMT [shelf.config.mpha:info]: All attached storage on the system is multipathed HA. The system has 6 disks assigned whereas it needs 3 to boot, will try to assign the required number.
Zero disks and install a new file system? y
This will erase all the data on the disks, are you sure? y
Zeroing disks takes about 255 minutes.
Tue Mar 20 09:02:31 GMT [coredump.spare.none:info]: No sparecore disk was found.
…………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………..
#启动控制器B,插上1块多的硬盘
Phoenix TrustedCore(tm) Server
Copyright 1985-2006 Phoenix Technologies Ltd. All Rights Reserved BIOS version: 5.1.1
Portions Copyright (c) 2007-2011 NetApp, Inc. All Rights Reserved
CPU = 1 Processors Detected, Cores per Processor = 2 Intel(R) Xeon(R) CPU E5220 @ 2.33GHz Testing RAM
512MB RAM tested 5120MB RAM installed 6144 KB L2 Cache
System BIOS shadowed
USB 2.0: MICRON eUSB DISK
BIOS is scanning PCI Option ROMs, this will take few seconds... +++++++++++++++++++
Boot Loader version 3.2
Copyright (C) 2000-2003 Broadcom Corporation. Portions Copyright (C) 2002-2008 NetApp
CPU Type: Intel(R) Xeon(R) CPU E5220 @ 2.33GHz
Starting AUTOBOOT press Ctrl-C to abort...
Autoboot of PRIMARY image aborted by user.
LOADER-B > autoboot
Loading X86_64/kernel/primary.krn:0x200000/49123928 0x30d9258/27512016 0x4b15f28/7980237 0x52b23f5/3 Entry at 0x00202018
Starting program at 0x00202018
MTRR0, will be reset to map region 3G to 4G as UC BBSP on this platform is 0x0000000000000180
ipmi_init_alloc_request_pool succeed (pool entries 0x80,size per entry 0xe0) ipmi_init0: DONE.
Press CTRL-C for special boot menu
IPMI:IPMI device rev. 2, firmware rev. 1.0, version 2.0
IPMI:Number of channels 0 ipmi_init1: DONE.
Special boot options menu will be available. Chelsio T3 RDMA Driver - version 0.1 Ipspace "iwarp-ipspace" created
Tue Mar 20 09:12:37 GMT [fci.initialization.failed:error]: Initialization failed on Fibre Channel adapter 0c. Tue Mar 20 09:12:37 GMT [fci.initialization.failed:error]: Initialization failed on Fibre Channel adapter 0d.
NetApp Release 7.3.6: Thu Jul 7 01:48:12 PDT 2011 Copyright (c) 1992-2011 NetApp.
Starting boot on Tue Mar 20 09:12:19 GMT 2012
Tue Mar 20 09:12:38 GMT [nvram.battery.turned.on:info]: The NVRAM battery is turned ON. It is turned OFF during system shutdown.
Tue Mar 20 09:12:43 GMT [diskown.isEnabled:info]: software ownership has been enabled for this system
(1) Normal boot.
(2) Boot without /etc/rc.
(3) Change password.
(4) Assign ownership and initialize disks for root volume.
(4a) Same as option 4, but create a flexible root volume.
(5) Maintenance mode boot.
Selection (1-5)? 5
Tue Mar 20 09:12:50 GMT [shelf.config.mpha:info]: All attached storage on the system is multi-pathed HA.
You have selected the maintenance boot option:
the system has booted in maintenance mode allowing the following operations to be performed:
? disk
fcadmin fcstat
sasadmin sasstat
acpadmin halt
help ifconfig
raid_config storage
sesdiag sysconfig
version vol
aggr sldiag
environment led_on
led_off sata
acorn scsi
disk_list ha-config
fcdiag fctest
disktest diskcopy
xortest faltest
outb disk_mung
Type "help
In a cluster, you MUST ensure that the partner is (and remains) down,
or that takeover is manually disabled on the partner node,
because clustering software is notstarted or fully enabled
in Maintenance mode.
FAILURE TO DO SO CAN RESULT IN YOUR FILESYSTEMS BEING DESTROYED Continue with boot? y
Tue Mar 20 09:12:53 GMT [netif.linkUp:info]: Ethernet c0b: Link up.
Tue Mar 20 09:12:54 GMT [iwarp-vfiler@ctrl.rdma.heartBeat:info]: Storage Interconnect status: Starting heartbeat to 192.168.2.235
Tue Mar 20 09:12:55 GMT [mgr.boot.reason_ok:notice]: System rebooted after a halt command.
Ipspace "acp-ipspace" created
Tue Mar 20 09:12:56 GMT [console_login_mgr:info]: root logged in from console *> disk show –v #查看磁盘的归属关系
Local System ID: 1574659048
DISK OWNER POOL SERIAL NUMBER ------------ ------------- ----- -------------
0a.00.3 (1574659072) Pool0 N000V8KL 0a.00.7 Not Owned NONE N000YG6L 0a
00.8 Not Owned NONE N002K7KL 0a.00.6 Not Owned NONE N000J0WL 0a.00.1 (1574659072) Pool0 N000J65L 0a.00.2 (1574659072) Pool0 N000J71L 0a.00.0 (1574659072) Pool0 J830SRPL 0a.00.5 (1574659072) Pool0 N000J5ML 0a.00.4 (1574659072) Pool0 N001YL7L *> disk assign 0a.00.7 #把磁盘0a.00.7分配给当前控制器
disk assign: Disk assigned but unable to obtain owner name. Re-run 'disk assign' with -o option to specify name.
Tue Mar 20 09:16:28 GMT [diskown.changingOwner:info]: changing ownership for disk 0a.00.7 (S/N N000YG6L) from unowned (ID -1) to (ID 1574659048)
*> disk assign 0a.00.6 #把磁盘0a.00.6分配给当前控制器
disk assign: Disk assigned but unable to obtain owner name. Re-run 'disk assign' with -o option to specify name.
Tue Mar 20 09:16:34 GMT [diskown.changingOwner:info]: changing ownership for disk 0a.00.6 (S/N N000J0WL) from
cifs setup
localhost/administrator
Qwe!@#123
install apache: http://mirrors.devlib.org/apache/httpd/binaries/win32/
aggr show_space -h
vol status -r # display volume and raid status
sysconfig -r # same as vol status -r
cifs CXQGUPC
fcp KFTVETB
iscsi OFYGUPC
nfs WDTITYG
snaprestore ECQRJMD
sv_ontap_pri MEDLPWA
(6)、aggr create aggrname
[ -f ]
[ -m ]
[ -n ]
[ -t raidtype ]
[ -r raidsize ]
[ -T disk-type ]
[ -R rpm ]
[ -L [compliance | enterprise] ]
[ -v ]
[ -l language-code ]
{ ndisks[@size] | -d disk1 [ disk2 ... ]
[ -d diskn [ diskn+1 ... ] ]
}
aggr create aggr3 -d v5.40 v5.41 v5.42 <———三块没有格式化的磁盘需要先格式化才加入aggr Creation of an aggregate with 3 disks has been initiated. The disks need to be zeroed before addition to the aggregate. The process has been initiated and you will be notified via the system log as disks are added.
Note however, that if system reboots before the disk zeroing is complete, the volume won't exist.
tan> aggr status –r <——正在格式化,此时状态是pending Aggregate aggr3 (creating, raid_dp, initializing) (zoned checksums) Plex /aggr3/plex0 (offline, empty, active)
Targeted to traditional volume or aggregate but not yet assigned to a raid group
RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks) --------- ------ ------------- ---- ---- ---- ----- -------------- --------------
pending v5.40 v5 2 8 FC:B 1 FCAL N/ A 520/1065984 527/1080448 (zeroing, 64% done)
pending v5.41 v5 2 9 FC:B 1 FCAL N/ A 520/1065984 527/1080448 (zeroing, 69% done)
pending v5.42 v5 2 10 FC:B 1 FCAL N/ A 520/1065984 527/1080448 (zeroing, 69% done)
创建一个新的aggr,-t选项指定raidtype(raid4,raiddp),-r选项指定aggr中RG组中最大的磁盘组,缺省的磁盘组和硬件平台有关。-T指定磁盘类型,比如:ATA, FCAL,LUN, SAS, SATA, and SCSI. 在一个aggr中不允许不同转速的磁盘放在一起。-T不能和-d一起用。
SAS-------- Serial Attached SCSI disks in matching shelves. SATA-------serial ATA disks in SAS shelves.
-m选项,在创建aggr的时候就指定是镜像的,指定的磁盘被分为两个plex,缺省创建的aggr不是镜像的。 -n选项,预演命令,不实际执行。
如果存储上有SnapLock Compliance or SnapLock Enterprise的license,则可以用-L选项来创建一个snaplock aggr.
(7)、aggr destroy { aggrname | plexname } [ -f ]
摧毁aggr,一旦执行上面的plex全部摧毁,要摧毁的aggr上不能有灵活卷(不管灵活卷的状态是什么),否则根本无法offline.
tan> aggr offline aggr2
aggr offline: Cannot offline aggregate 'aggr2' because it contains one or more flexible volumes
aggr摧毁后,它里面的磁盘就变成spare盘了;只有offline的aggr、传统卷和plex可以被摧毁。
(8)、aggr media_scrub status [ aggrname | plexname | groupname] [ -v ]
tan> aggr media_scrub status aggr1
aggr media_scrub /aggr1/plex0/rg0 is 70% complete
(9)、aggr mirror aggrname [ -f ] [ -n ] [ -v victim_aggrname ] [ -d disk1 [ disk2 ... ] ] 把非镜像的aggr转为镜像的aggr
tan> aggr mirror aggr1 -d v5.28,v5.29,v5.43,v5.16,v5.17,v5.18
aggr mirror: Couldn't mirror volume: Operation requires a syncmirror_local license. You must license it and then reboot.
tan> aggr mirror aggr1 -f -d v5.28 v5.29 v5.43 v5.16 v5.17 v5.18 这是做完镜像后两个plex的状态。
fcadmin config -t initiator 0b
fcadmin config -t initiator 0d
lunsetup
do you want to create a LUN? y
multiprotocol type of LUN: LINUX
enter LUN path: /vol/fcpvol/fcp.lun
do you want to the LUN to be space reserved: y
enter LUN size: 3384g
enter comment string:
name of initiator group: initiatorFCP
type of initiator group initiatorFCP (FCP/ISCSI): FCP
ENTER COMMA SEPARATED PORTNAMES:
For more detail about setup Lun/VOL/AGGR, please refer Lun on linux