/sys/dev/bnxt: Enable NPAR support on BCM57504

[viniciusferrao]

This commit enables NPAR support for Broadcom 57504 10/25GbE NICs.

I've tested this patch against a live 14.1-RELEASE system that were initially unable to detect a driver for the cards and after the changes it detected and worked as expected.

That's the results:

pciconf -lv:

bnxt0@pci0:0:8:0:	class=0x020000 rev=0x12 hdr=0x00 vendor=0x14e4 device=0x1801 subvendor=0x17aa subdevice=0x4050
    vendor     = 'Broadcom Inc. and subsidiaries'
    device     = 'BCM57504 NetXtreme-E Ethernet Partition'
    class      = network
    subclass   = ethernet
bnxt1@pci0:0:9:0:	class=0x020000 rev=0x12 hdr=0x00 vendor=0x14e4 device=0x1801 subvendor=0x17aa subdevice=0x4050
    vendor     = 'Broadcom Inc. and subsidiaries'
    device     = 'BCM57504 NetXtreme-E Ethernet Partition'
    class      = network
    subclass   = ethernet

dmesg:

bnxt0: <Broadcom BCM57504 NetXtreme-E Ethernet Partition> mem 0x500f51a0000-0x500f51affff,0x500f3000000-0x500f3ffffff,0x500f51c8000-0x500f51cffff irq 22 at device 9.0 on pci0
bnxt0: Using 256 TX descriptors and 256 RX descriptors
bnxt0: Using 4 RX queues 4 TX queues
bnxt0: Using MSI-X interrupts with 5 vectors
bnxt0: Link is UP full duplex, FC - none - 10000 Mbps 
bnxt0: Ethernet address: 00:62:0b:66:a9:99
bnxt0: netmap queues/slots: TX 4/256, RX 4/256
bnxt0: link state changed to UP
bnxt1: <Broadcom BCM57504 NetXtreme-E Ethernet Partition> mem 0x500f51b0000-0x500f51bffff,0x500f4000000-0x500f4ffffff,0x500f51d0000-0x500f51d7fff irq 25 at device 10.0 on pci0
bnxt1: Using 256 TX descriptors and 256 RX descriptors
bnxt1: Using 4 RX queues 4 TX queues
bnxt1: Using MSI-X interrupts with 5 vectors
bnxt1: Link is UP full duplex, FC - none - 10000 Mbps 
bnxt1: Ethernet address: 00:62:0b:66:a9:98
bnxt1: netmap queues/slots: TX 4/256, RX 4/256
bnxt1: link state changed to UP

bnxt0: Link is UP full duplex, FC - none - 10000 Mbps 
bnxt1: Link is UP full duplex, FC - none - 10000 Mbps 

ifconfig:

bnxt0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
	options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
	ether 00:62:0b:66:a9:99
	media: Ethernet autoselect (Unknown <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bnxt1: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
	options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
	ether 00:62:0b:66:a9:98
	inet 10.20.0.244 netmask 0xffffff00 broadcast 10.20.0.255
	media: Ethernet autoselect (Unknown <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Also done some performance tests and it seems good for a non tuned interface:

Connecting to host 10.20.0.101, port 5201
[  5] local 10.20.0.244 port 59365 connected to 10.20.0.101 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.01   sec  1.01 GBytes  8.53 Gbits/sec   37    231 KBytes       
[  5]   1.01-2.01   sec   814 MBytes  6.87 Gbits/sec   29    213 KBytes       
[  5]   2.01-3.01   sec   948 MBytes  7.89 Gbits/sec   50    110 KBytes       
[  5]   3.01-4.01   sec   911 MBytes  7.71 Gbits/sec   26    267 KBytes       
[  5]   4.01-5.02   sec  1.01 GBytes  8.63 Gbits/sec  101    115 KBytes       
[  5]   5.02-6.02   sec   883 MBytes  7.41 Gbits/sec   20   64.9 KBytes       
[  5]   6.02-7.02   sec   774 MBytes  6.47 Gbits/sec   14    162 KBytes       
[  5]   7.02-8.04   sec  1.11 GBytes  9.36 Gbits/sec   18    233 KBytes       
[  5]   8.04-9.02   sec  1.02 GBytes  8.94 Gbits/sec   21    301 KBytes       
[  5]   9.02-10.03  sec  1.05 GBytes  9.00 Gbits/sec   16   53.4 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.03  sec  9.43 GBytes  8.08 Gbits/sec  332             sender
[  5]   0.00-10.08  sec  9.43 GBytes  8.04 Gbits/sec                  receiver

iperf Done.

If there's any test that I should perform or if I should open this PR through other FreeBSD systems please let me know.

Thanks.

[rilysh]

Looks good!

[bsdimp]

This looks good to me, will ping the Broadcom folks.

[sumitsaxena11]

Looks good to me.

[viniciusferrao]

Regarding the failed workflows due to computing limits, is there anything that I should do?

[bsdimp]

Regarding the failed workflows due to computing limits, is there anything that I should do?

Sorry I missed this due to the 4th here in the us. No. there's nothing you can do.