智能网卡Stingray Setup

硬件型号

BCM958802A8048C 2-port 25Gb 8GB DDR4 SmartNIC

Host侧配置

Setup on Host

1.Install bnxt_en.ko

$ modprobe vxlan
$ insmod bnxt_en.ko

2.Bring up PF8(the first PF on the host) and assign an IP address 192.168.1.20

$ ifconfig enp8s0f0 192.168.1.20

Now the host can SSH to the PS225 default IP address: 192.168.1.10 to access the Linux shell on SOC.

$ ssh 

编译并安装bnxt_en driver

git clone  . &&

git checkout bcm/stable && cd bnxt_en/main/Cumulus/drivers/linux/v3 &&

make && insmod bnxt_en.ko

升级the PS225 Software

The PS225 cards come out of manufacturing with a default 8 + 8PF configuration. The default configurations do not support SR-IOV or pairing models. It`s necessary to enable SR-IOV by upgrading PS225 software and Nitro Configurations(Refer to: "5880X-PS225-UG1xx" Appendix C ).

安装 TFTP Server

1.Install following packages.

$ sudo apt-get install xinetd tftpd tftp

2.Create /etc/xinetd.d/tftp and put this entry

service tftp
{
protocol        = udp
port            = 69
socket_type     = dgram
wait            = yes
user            = nobody
server          = /usr/sbin/in.tftpd
server_args     = /tftpboot
disable         = no
}

3. Create a folder /tftpboot  this should match whatever you gave in server_args. mostly it will be tftpboot

$ sudo mkdir /tftpboot
$ sudo chmod -R 777 /tftpboot
$ sudo chown -R nobody /tftpboot

4.Restart the xinetd service.

$ sudo /etc/init.d/xinetd stop
$ sudo /etc/init.d/xinetd start

下载binaries

Download binaries according to the corresponding hardware model, such as:

/

Notes:

Stingray binaries manage by Git LFS, Please refer to  for setting up Git LFS.

升级Software

1.From the host SSH to SOC

$ ssh 

2.Copy binary tarball to /tftpboot and uncompress

3.Run the update script

update-me.sh -s 192.168.1.20 -d /tftpboot -i all -t tftp

4. Reboot PS225

The reboot must be applied on the SOC, I think it's best to restart host as well.

升级Nitro Configurations

34 Nitro configurations are provided by Broadcom.  

root@ ls ./nitro/extra_cfgs/
bcm958802a8048_1x10g.pkg bcm958802a8048_2x10g.pkg bcm958802a8048_2x25g_8+5_virtio_fwd.pkg bcm958802a8048_2xAN_8+5_virtio_fwd.pkg
bcm958802a8048_1x25g.pkg bcm958802a8048_2x25g_8+1_baremetal.pkg bcm958802a8048_2x25g.pkg bcm958802a8048_2xAN.pkg
bcm958802a8048_1xAN.pkg bcm958802a8048_2x25g_8+2_baremetal_nofec.pkg bcm958802a8048_2xAN_8+1_baremetal.pkg bcm958802a8048_8+8_pf_rdma.pkg
bcm958802a8048_2x10g_8+2_baremetal.pkg bcm958802a8048_2x25g_8+2_baremetal.pkg bcm958802a8048_2xAN_8+2_baremetal.pkg BCM958802A8048C.pkg
bcm958802a8048_2x10g_8+3_pf.pkg bcm958802a8048_2x25g_8+3_pf.pkg bcm958802a8048_2xAN_8+3_pf.pkg stingray_1p.pkg
bcm958802a8048_2x10g_8+4_pf.pkg bcm958802a8048_2x25g_8+4_pf.pkg bcm958802a8048_2xAN_8+4_pf.pkg stingray_2p.pkg
bcm958802a8048_2x10g_8+4_pf_rep.pkg bcm958802a8048_2x25g_8+4_pf_rep.pkg bcm958802a8048_2xAN_8+4_pf_rep.pkg stingray_4p.pkg
bcm958802a8048_2x10g_8+5_pf_rep.pkg bcm958802a8048_2x25g_8+5_pf_rep.pkg bcm958802a8048_2xAN_8+5_pf_rep.pkg
bcm958802a8048_2x10g_8+5_virtio_fwd.pkg bcm958802a8048_2x25g_8+5_virtio_fwd_ncsi_smbus.pkg bcm958802a8048_2xAN_8+5_virtio_fwd_ncsi_smbus.pkg

You must read "Appendix C" in "5880X-PS225-UG1xx"  carefully to choose what you want. 

To load the .pkg with following commands:

update-me.sh -s 192.168.1.20 -d /tftpboot -t tftp -i nitro -c  bcm958802a8048_2xAN_8+4_pf_rep

Reboot PS225: The reboot must be applied on the SOC, I think it's best to restart host as well.