在RHEL 6.5linux下关于单张卡单口 每台服务器共两张卡，以下配置经过IB 交换机配置如下：
1、首先从配置文件/etc/rdma/opensm.conf复制出2个配置文件/etc/rdma/opensm.conf.1和/etc/rdma/opensm.conf.2
2、在这两个配置文件中修改以下两个参数guid= 0x24be05ffffb521c1（port口的GUID）可以通过ibstat命令查看，sm_priority=14 (按照次序排列)第一对链路设置为sm_priority=15，第二对链路sm_priority=14。
3、重启opensm服务 server opensm restart
4、在网络测试时候通过ibping命令实现
[root@xmutdb1 rdma]# ibping -v -C mlx4_0 -G 0xe41d2d0300dc5a61
Pong from xmutdb2.(none) (Lid 3): time 0.139 ms
Pong from xmutdb2.(none) (Lid 3): time 0.092 ms
Pong from xmutdb2.(none) (Lid 3): time 0.089 ms
^C
--- xmutdb2.(none) (Lid 3) ibping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2370 ms
rtt min/avg/max = 0.089/0.106/0.139 ms
[root@xmutdb1 rdma]#
[root@xmutdb2 sysconfig]# ibping -S -C mlx4_0 //在目标节点启动Server端模式
5、链路信息查看
[root@xmutdb2 sysconfig]# iblinkinfo -C mlx4_1 //只有指定卡才能看到所有链路
CA: xmutdb1 mlx4_1:
0xe41d2d0300dc4641 1 1[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==> 2 3[ ] "MF0;switch-e67c5c:SX6012/U1"
( )Switch: 0xf452140300744ae0 MF0;switch-e67c5c:SX6012/U1:
2 1[ ] ==( Down/ Polling)==> [ ] "" ( )
2 2[ ] ==( Down/ Polling)==> [ ] "" ( )
2 3[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==> 1 1[ ] "xmutdb1 mlx4_1" ( )
2 4[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==> 4 1[ ] "xmutdb2 mlx4_1" ( )
2 5[ ] ==( Down/ Polling)==> [ ] "" ( )
2 6[ ] ==( Down/ Polling)==> [ ] "" ( )
2 7[ ] ==( Down/ Polling)==> [ ] "" ( )
2 8[ ] ==( Down/ Polling)==> [ ] "" ( )
2 9[ ] ==( Down/ Polling)==> [ ] "" ( )
2 10[ ] ==( Down/ Polling)==> [ ] "" ( )
2 11[ ] ==( Down/ Polling)==> [ ] "" ( )
2 12[ ] ==( Down/ Polling)==> [ ] "" ( )
CA: xmutdb2 mlx4_1:
0xe41d2d0300dc5cb1 4 1[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==> 2 4[ ] "MF0;switch-e67c5c:SX6012/U1"
( )[root@xmutdb2 sysconfig]#
[root@xmutdb2 sysconfig]# iblinkinfo -C mlx4_0
CA: xmutdb1 mlx4_0:
0xe41d2d0300dc4661 5 1[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==> 6 11[ ] "MF0;switch-e6825c:SX6012/U1"
( )Switch: 0xf4521403006e3680 MF0;switch-e6825c:SX6012/U1:
6 1[ ] ==( Down/ Polling)==> [ ] "" ( )
6 2[ ] ==( Down/ Polling)==> [ ] "" ( )
6 3[ ] ==( Down/ Polling)==> [ ] "" ( )
6 4[ ] ==( Down/ Polling)==> [ ] "" ( )
6 5[ ] ==( Down/ Polling)==> [ ] "" ( )
6 6[ ] ==( Down/ Polling)==> [ ] "" ( )
6 7[ ] ==( Down/ Polling)==> [ ] "" ( )
6 8[ ] ==( Down/ Polling)==> [ ] "" ( )
6 9[ ] ==( Down/ Polling)==> [ ] "" ( )
6 10[ ] ==( Down/ Polling)==> [ ] "" ( )
6 11[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==> 5 1[ ] "xmutdb1 mlx4_0" ( )
6 12[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==> 3 1[ ] "xmutdb2 mlx4_0" ( )
CA: xmutdb2 mlx4_0:
0xe41d2d0300dc5a61 3 1[ ] ==( 4X 14.0625 Gbps Active/ LinkUp)==> 6 12[ ] "MF0;switch-e6825c:SX6012/U1"
( )[root@xmutdb2 sysconfig]#
[root@xmutdb1 rdma]# sminfo -C mlx4_0
sminfo: sm lid 5 sm guid 0xe41d2d0300dc4661, activity count 1266 priority 15 state 3 SMINFO_MASTER
[root@xmutdb1 rdma]# sminfo -C mlx4_1
sminfo: sm lid 1 sm guid 0xe41d2d0300dc4641, activity count 1258 priority 14 state 3 SMINFO_MASTER
[root@xmutdb1 rdma]#
阅读(2981) | 评论(0) | 转发(0) |
