Netapp Network VIF Setup

Netapp Network VIF Setup

一.vif接口概念

netapp把端口聚合叫做vif(与术语port trunk/etherchanell/link aggregate是同一个意思),支持以下三中聚合方式:

a.single-mode,端口工作在active和standby状态,只有一个端口处于actvie状况,当active端口失效,系统自动从standby中选择一个端口成为active口;不需要交换机支持aggre,也不需要在交换机上做任何相关配置.所有端口具有相同的ＭＡＣ地址(如下所示)

 

b.multi-mode(static),与IEEE 802.3ad(static)兼容,所有端口处于active状态,具有相同的mac地址;能够检测端口的链路状态,但不能数据的丢失;交换机需要支持LACP;在multimode(包括static和dynamic)下,存储系统对outbound traffic支持三种load-balance算法:IP-based/MAC-based/Round robin;

c.multi-mode(dynamic),与IEEE 802.3ad(dynamic)兼容,即lLACP协议(cisco叫做PAgP),所有端口均处于active状态,具有相同的mac地址;交换机需要支持LACP;在lacp下,vif接口必须配置为first-level,并且LB应该配置为IP-based(这也是默认方式);在lacp下,存储系统不仅能检测链路的丢失,还能检测数据流的丢失,因此该模式与HA环境相兼容;

 

二.vif接口配置

使用vif命令创建删除vif接口,添加或删除物理网口到vif中,查看vif接口状态;

最多支持16个物理口聚合成1个vif接口;所有物理口需具有相同的MTU;不能将vlan接口添加到vif中;支持10/100/1000m网卡;

对于10G TOE卡有以下限制:multimode支持2个物理口,LACP不支持,TOE功能会被禁用;

 

nasim*> vif
Usage:
        vif create [single|multi|lacp] -b [rr|mac|ip] []    创建vif接口,指定模式/接口名称/LB算法/相应的物理接口;
        vif add      将物理口添加到以存在的vif接口中;                                  
        vif delete 从vif接口中删除物理接口;
        vif destroy               删除vif逻辑接口;
        vif {favor|nofavor}    在接口组中,选择active接口;
        vif status []             查看vif接口状态;
        vif stat [interval]     查看vif流量统计信息;

1.single vif配置

step1:使用vif create命令创建single vif逻辑接口:

nasim*> vif create single singletrunk1 ns0 ns1
nasim*>

查看系统接口信息,
nasim*> ifconfig -a
ns0: flags=808042 mtu 1500
        ether 02:50:56:1d:ff:f7 (Linux AF_PACKET socket)
        trunked singletrunk1
ns1: flags=808042 mtu 1500
        ether 02:50:56:1d:ff:f7 (Linux AF_PACKET socket)
        trunked singletrunk1
lo: flags=1948049 mtu 9188
        inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
singletrunk1: flags=8042 mtu 1500
        ether 02:50:56:1d:ff:f7 (Disabled virtual interface)
nasim*>

step2:给虚拟接口配置IP地址:

nasim*>
nasim*> ifconfig singletrunk1 192.168.1.20 netmask 255.255.255.0
nasim*> Mon Jan 18 04:35:58 GMT [pvif.switchLink:warning]: singletrunk1: switching to ns1 (系统随机选择ns1口为active接口)
Mon Jan 18 04:35:58 GMT [iscsi.notice:notice]: ISCSI: New session from initiator iqn.2005-03.com.redhat:01.74394c92c697 at IP addr 192.168.1.60

nasim*> ifconfig -a
ns0: flags=848043 mtu 1500
        ether 02:50:56:1d:ff:f7 (Linux AF_PACKET socket)
        trunked singletrunk1
ns1: flags=848043 mtu 1500
        ether 02:50:56:1d:ff:f7 (Linux AF_PACKET socket)
        trunked singletrunk1
lo: flags=1948049 mtu 9188
        inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
singletrunk1: flags=848043 mtu 1500
        inet 192.168.1.20 netmask 0xffffff00 broadcast 192.168.1.255
        ether 02:50:56:1d:ff:f7 (Enabled virtual interface)
nasim*>

step3:手动选择active接口,查看vif接口状态:

nasim*>
nasim*> vif favor ns0
nasim*>    
nasim*> vif status singletrunk1 (在singlemode下,只有一个物理口为up,即active,其他为down,即standby)
default: transmit 'IP Load balancing', VIF Type 'multi_mode', fail 'log'
singletrunk1: 1 link, transmit 'none', VIF Type 'single_mode' fail 'default'
         VIF Status     Up      Addr_set
        up:
        ns1: state up, since 18Jan2010 05:19:38 (00:55:06)
                mediatype: Linux AF_PACKET socket
                flags: enabled
                input packets 0, input bytes 0
                output packets 562, output bytes 534476
                output probe packets 0, input probe packets 0
                up indications 1, broken indications 0
                drops (if) 0, drops (link) 0
                indication: up at 18Jan2010 05:19:38
                        consecutive 3304, transitions 1
        down:
        ns0: state down, since 18Jan2010 05:19:38 (00:55:06)
                mediatype: Linux AF_PACKET socket
                flags: enabled
                input packets 0, input bytes 0
                output packets 0, output bytes 0
                output probe packets 0, input probe packets 0
                up indications 1, broken indications 0
                drops (if) 0, drops (link) 0
                indication: up at 18Jan2010 05:19:38
                        consecutive 3304, transitions 1

nasim*> Mon Jan 18 04:44:02 GMT [pvif.switchLink:warning]: singletrunk1: switching to ns0

step4:验证linux iscsi initiator连接

nasim*> iscsi session show
No active sessions

没有活动iscsi会话,查看主机端启动日志:

[root@rhe5 log]# grep iscsi /var/log/messages
Jan 17 03:42:20 rhe5 iscsid: connect failed (113)
Jan 17 04:36:52 rhe5 iscsid: Portal group tag mismatch, expected 1000, received 2000
Jan 17 04:36:57 rhe5 iscsid: connect failed (113)
Jan 17 05:04:11 rhe5 kernel: iscsi: cmd 0x28 is not queued (8)
Jan 17 05:04:11 rhe5 kernel: iscsi: cmd 0x28 is not queued (8)
Jan 17 05:04:14 rhe5 iscsid: connect failed (113)
Jan 17 05:04:41 rhe5 iscsid: connect failed (113)
Jan 17 05:04:47 rhe5 iscsid: connect failed (113)
Jan 17 05:06:08 rhe5 kernel: Loading iSCSI transport class v2.0-685.<5>iscsi: registered transport (tcp)
Jan 17 05:06:08 rhe5 kernel: iscsi: registered transport (iser)
Jan 17 05:06:08 rhe5 iscsid: connection0:0 is operational now
Jan 17 05:06:08 rhe5 iscsid: picking unique OUI for the same target node name iqn.1992-08.com.netapp:sn.99900641
Jan 17 05:06:09 rhe5 iscsid: connection1:0 is operational now
Jan 17 05:12:01 rhe5 iscsid: connect failed (113)
Jan 17 05:12:01 rhe5 iscsid: connect failed (113)
Jan 17 05:12:04 rhe5 iscsid: Portal group tag mismatch, expected 1000, received 2000
Jan 17 05:12:06 rhe5 iscsid: connect failed (113)
Jan 17 05:13:39 rhe5 kernel:  session0: iscsi: session recovery timed out after 120 secs
Jan 17 05:13:39 rhe5 iscsid: connect failed (113)
Jan 17 05:21:23 rhe5 iscsid: connect failed (113)
Jan 17 21:23:19 rhe5 iscsid: connect failed (113)
Jan 17 21:23:25 rhe5 iscsid: connect failed (113)
Jan 17 21:25:10 rhe5 kernel: Loading iSCSI transport class v2.0-685.<5>iscsi: registered transport (tcp)
Jan 17 21:25:10 rhe5 kernel: iscsi: registered transport (iser)
Jan 17 21:25:14 rhe5 iscsid: connect failed (113)
Jan 17 21:25:15 rhe5 iscsid: Portal group tag mismatch, expected 1000, received 2000 

上述错误信息表明,portal group tag不匹配导致iscsi login 失败;查看存储设备和主机上的配置:

[root@rhe5 log]# cd /var/lib
[root@rhe5 lib]# cd iscsi
[root@rhe5 iscsi]# ls
nodes  send_targets
[root@rhe5 iscsi]# cd nodes
[root@rhe5 nodes]# ls
iqn.1992-08.com.netapp:sn.99900641
[root@rhe5 nodes]# cd iqn.1992-08.com.netapp\:sn.99900641/
[root@rhe5 iqn.1992-08.com.netapp:sn.99900641]# ls
192.168.1.20,3260  192.168.1.21,3260
[root@rhe5 iqn.1992-08.com.netapp:sn.99900641]# more 192.168.1.20,3260
node.name = iqn.1992-08.com.netapp:sn.99900641
node.transport_name = tcp
node.tpgt = 1000
node.active_conn = 1
node.startup = automatic
node.session.initial_cmdsn = 0
node.session.auth.authmethod = None
node.session.timeo.replacement_timeout = 120
……

nasim*> iscsi portal show
Network portals:
IP address        TCP Port  TPGroup  Interface
192.168.1.20         3260    2000    singletrunk1

修改主机上portal tag与存储设备上一致,重启主机;

[root@rhe5 send_targets]# fdisk -l

Disk /dev/sda: 23.6 GB, 23622320128 bytes
255 heads, 63 sectors/track, 2871 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1           6       48163+  83  Linux
/dev/sda2               7         515     4088542+  83  Linux
/dev/sda3             516        2741    17880345   83  Linux
/dev/sda4            2742        2871     1044225    5  Extended
/dev/sda5            2742        2871     1044193+  82  Linux swap / Solaris

Disk /dev/sdb: 36 MB, 36700160 bytes
2 heads, 35 sectors/track, 1024 cylinders
Units = cylinders of 70 * 512 = 35840 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1024       35822+  83  Linux
[root@rhe5 send_targets]# cd /
[root@rhe5 /]# mkdir /mnt/nalun
[root@rhe5 /]# mount /dev/sdb1 /mnt/nalun

nasim*>
nasim*> iscsi session show
Session 5
  Initiator Information
    Initiator Name: iqn.2005-03.com.redhat:01.74394c92c697
    ISID: 00:02:3d:00:00:00
    Initiator Alias: rhe5

2.multimode vif配置

nasim*>
nasim*> vif destroy singletrunk1
vif: cannot destroy a pvif that is up
nasim*>
nasim*>
nasim*> ifconfig singletrunk1 down
nasim*> Mon Jan 18 06:17:29 GMT [pvif.vifConfigDown:info]: singletrunk1: Configured down

nasim*> vif destroy singletrunk1
nasim*>
nasim*>
nasim*> ifconfig -a
ns0: flags=808042 mtu 1500
        ether 00:50:56:1d:ff:f7 (Linux AF_PACKET socket)
ns1: flags=808042 mtu 1500
        ether 00:50:56:1e:ff:f7 (Linux AF_PACKET socket)
lo: flags=1948049 mtu 9188
        inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
nasim*>
nasim*>
nasim*> vif create multi multitrunk1 ns0 ns1
nasim*>
nasim*> ifconfig multitrunk1 192.168.1.20 netmask 255.255.255.0
nasim*> Mon Jan 18 06:18:45 GMT [pvif.switchLink:warning]: multitrunk1: switching to ns1
Mon Jan 18 06:18:48 GMT [iscsi.notice:notice]: ISCSI: New session from initiator iqn.2005-03.com.redhat:01.74394c92c697 at IP addr 192.168.1.60

nasim*>
nasim*> vif status multitrunk1  (在multimode下,vif接口组中所有物理接口都为up状态,即active状况)
default: transmit 'IP Load balancing', VIF Type 'multi_mode', fail 'log'
multitrunk1: 2 links, transmit 'IP Load balancing', VIF Type 'multi_mode' fail 'default'
         VIF Status     Up      Addr_set
        up:
        ns1: state up, since 18Jan2010 06:18:45 (00:03:01)
                mediatype: Linux AF_PACKET socket
                flags: enabled
                input packets 0, input bytes 0
                output packets 37, output bytes 2938
                up indications 1, broken indications 0
                drops (if) 0, drops (link) 0
                indication: up at 18Jan2010 06:18:45
                        consecutive 180, transitions 1
        ns0: state up, since 18Jan2010 06:18:45 (00:03:01)
                mediatype: Linux AF_PACKET socket
                flags: enabled
                input packets 0, input bytes 0
                output packets 119, output bytes 63710
                up indications 1, broken indications 0
                drops (if) 0, drops (link) 0
                indication: up at 18Jan2010 06:18:45
                        consecutive 180, transitions 1

nasim*> iscsi session show
Session 6
  Initiator Information
    Initiator Name: iqn.2005-03.com.redhat:01.74394c92c697
    ISID: 00:02:3d:00:00:00
    Initiator Alias: rhe5

3.lacp vif配置

nasim*>
nasim*> ifconfig multitrunk1 down
nasim*> Mon Jan 18 06:31:28 GMT [pvif.vifConfigDown:info]: multitrunk1: Configured down

nasim*>
nasim*> vif destroy multitrunk1
nasim*>
nasim*> vif create lacp lacptrunk1 ns0 ns1
nasim*>
nasim*>
nasim*> ifconfig lacptrunk1 192.168.1.20 netmask 255.255.255.0 up
nasim*>
nasim*> ifconfig -a
ns0: flags=848043 mtu 1500
        ether 02:50:56:1d:ff:f7 (Linux AF_PACKET socket)
        trunked lacptrunk1
ns1: flags=848043 mtu 1500
        ether 02:50:56:1d:ff:f7 (Linux AF_PACKET socket)
        trunked lacptrunk1
lo: flags=1948049 mtu 9188
        inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
lacptrunk1: flags=48043 mtu 1500
        inet 192.168.1.20 netmask 0xffffff00 broadcast 192.168.1.255
        ether 02:50:56:1d:ff:f7 (Disabled virtual interface)
nasim*>                  
nasim*>
nasim*>
nasim*> ifconfig lacptrunk1 up
nasim*>
nasim*>
nasim*> vif status lacptrunk1  ()
default: transmit 'IP Load balancing', VIF Type 'multi_mode', fail 'log'
lacptrunk1: 0 links, transmit 'IP Load balancing', VIF Type 'lacp' fail 'default'
         VIF Status     Addr_set
        lag_inactive:
        ns1: state lag_inactive, since 18Jan2010 06:32:21 (00:00:50)
                mediatype: Linux AF_PACKET socket
                flags: lacp enabled
                input packets 0, input bytes 0
                input lacp packets 0, output lacp packets 2
                output packets 2, output bytes 248
                up indications 1, broken indications 0
                drops (if) 0, drops (link) 0
                indication: lag_inactive at 18Jan2010 06:32:21
                        consecutive 0, transitions 1
        ns0: state lag_inactive, since 18Jan2010 06:32:21 (00:00:50)
                mediatype: Linux AF_PACKET socket
                flags: lacp enabled
                input packets 0, input bytes 0
                input lacp packets 0, output lacp packets 2
                output packets 2, output bytes 248
                up indications 1, broken indications 0
                drops (if) 0, drops (link) 0
                indication: lag_inactive at 18Jan2010 06:32:21
                        consecutive 0, transitions 1 

nasim*>
nasim*> ping 192.168.1.20
ping: wrote 192.168.1.20 64 chars, error=No route to host
ping: wrote 192.168.1.20 64 chars, error=No route to host
nasim*>
nasim*> route -s
Routing tables

Internet:
Destination      Gateway            Flags     Refs     Use  Interface
localhost        localhost          UH          0        0  lo

由于交换机不支持lacp,lacp vif配置不成功