I still cannot use a single 2-port QSFP+ NIC to connect to 2x N321s. Using the Intel tools to set the NIC to 2x2x10, the NIC doesn't recognize the second physical port as a valid connection to a USRP, but works as a loopback or a connection to another NIC. I asked Intel regarding the issues to see if we were configuring the NIC incorrectly, and this is the response we got:
*Response from Intel on XL710-QDA2* “When configured for 2x2x10 the lanes are split between the ports. The A configuration utilizes lanes 1 and 2 on the top port and lanes 3 and 4 on the bottom port. These correlate to the physical connections on the QSFP+ module. If you are able to configure the port of the other device to utilize lanes 3 and 4, then it should be able to connect to the bottom port in the A configuration.” *Problem:* XQ FPGA image doesn’t accept connections from lanes 2 and 3, just 0 and 1. This prevents the bottom port on the Intel NIC from working in the 2x2x10 configuration: InterfaceHGXGWXXQAQ SFP+ 0 1 GbE 10 GbE White Rabbit White Rabbit 10 GbE SFP+ 1 10 GbE 10 GbE 10 GbE Unused 10 GbE QSFP+ lane 0 Unused Unused Unused 10 GbE Aurora QSFP+ lane 1 Unused Unused Unused 10 GbE Aurora QSFP+ lane 2 Unused Unused Unused *Unused* Aurora QSFP+ lane 3 Unused Unused Unused *Unused* Aurora In the 2 N321 configuration, lane 2 and 3 are valid, but point to a different USRP. I'm hoping theres a UHD or FW change that can prevent the QSFP->2x10G breakout cable because that defeats the purpose of using a QSFP+ NIC instead of a 4-port 10G NIC (Intel X710-DA4) Thanks, Pat On Tue, Nov 24, 2020 at 1:01 PM Michael Dickens <michael.dick...@ettus.com> wrote: > Hi Pat - I'm glad that info helped! > > Yes, I plan on adding this information into the N32x Getting Started > Guide, once I have a better handle on it. Right now I have just a few data > points & those are not consistent! and I don't know why! Thus ... > > Which Intel QSFP+ utility did you end up using? There are 2 that I can > find: > > 1) EPCT: > https://downloadcenter.intel.com/download/28933/Intel-Ethernet-Port-Configuration-Tool-Linux- > > This is the newer version that seems to work. > > 2) QCU : > https://downloadcenter.intel.com/download/25851/Intel-QSFP-Configuration-Utility-Linux-Final-Release?product=46828 > > This one is deprecated, though it still works to some extent. > > === > > When I execute (A), I get the following options: "4x10" and "2x2x10". I do > not get an "A" or "B" or "LOM" or whatever. Just literally those 2 options. > > I think the first one means "1x(4x10)", meaning that just port0 is active > & provides 4 data lates. I was hoping this option would work with a 1:4 > SFP+ breakout cable from FS.com, but to the best of my testing I can get > just 1 of those 4 SFP+ links to work. Supposedly if one uses the Intel 1:4 > breakout cable this will work ... but that's paying $350 for an otherwise > $50 cable! I'm still investigating here. Ideally this NIC would provide > "2x(4x10)" with 2 1:4 breakout cables, to get double the SFP+ density of > current NICs (e.g., the X710-DA4 ort X722-DA4). > > The second one implies to me that both ports are available & providing 2 > data lanes each. The best I've been able to do is use "2x2x10" with port0; > port1 doesn't seem to be working in this setting. > > Admittedly, I might need to update to the current Intel Linux drivers for > the XL710 NIC. I usually let the OS handle this for me -- in this case, > Ubuntu 20.04 latest. There are new Intel drivers from early November 2020, > but I don't think the XL710 had any updates from the prior version. > > I'm curious what driver version & OS / version you're using ... maybe > let's catch up off-list for a bit & see what we can figure out here. > Cheers! - MLD > > On Tue, Nov 24, 2020 at 9:06 AM Patrick Kane <prkan...@gmail.com> wrote: > >> Hi Mike, >> >> That seemed to do the trick, thanks for info! At some point, can we make >> these steps part of the N32x getting started docs? >> >> Also- the config utilty makes me choose 2x2x10 A, B, or LOM. Choosing A >> disables the second port on the QDA2, and B disables the first port. LOM >> disables both ports (expected because it's not a MB NIC). My ideal case is >> using 2x N321s over QSFP on the same XL710-QDA2 NIC. Have you had any luck >> in this configuration? >> >> Thanks, >> Pat >> >> On Mon, Nov 23, 2020 at 9:23 PM Michael Dickens < >> michael.dick...@ettus.com> wrote: >> >>> Hi Pat - I recently verified that the N321 QAFP+ interface works with >>> UHD 4.0 release. I am also using an Intel XL710 (QDA2, but that probably >>> doesn’t matter too much). The trick for me was using the Intel QSFP+ NIC >>> configuration tool to set the NIC to 2x(2x10 Gb) mode. This is the setting >>> that the N321 requires, and one that the NIC provides. Once that was set >>> then you need to configures the host and USRP network interfaces as you >>> normally would. After all of that, the link worked very nicely! I hope this >>> is useful! - MLD >>> >>> On Nov 23, 2020, at 4:44 PM, Patrick Kane via USRP-users < >>> usrp-users@lists.ettus.com> wrote: >>> >>> >>> >>> I have an N321 connected to serial console and QSFP+ through a XL710 >>> Intel NIC. With the default HG image, I can connect through 1G and serial >>> as expected. I updated the filesystem to UHD 4.0.0.0 using mender, and the >>> build artifact reflects that this was successful. Then, after loading the >>> XQ image (using 2x 10Gb lanes through QSFP+ port), I lose all ethernet >>> connectivity through the 1G port SFP0 (expected), but I get the following >>> output in the console window: >>> >>> >>> [ 451.560674] nixge 40000000.ethernet sfp0: Link is Up - 10Gbps/Full - >>> flow control off >>> >>> [ 453.800673] nixge 40000000.ethernet sfp0: Link is Down >>> >>> [ 454.920676] nixge 40000000.ethernet sfp0: Link is Up - 10Gbps/Full - >>> flow control off >>> >>> [ 458.280672] nixge 40000000.ethernet sfp0: Link is Down >>> >>> [ 459.400677] nixge 40000000.ethernet sfp0: Link is Up - 10Gbps/Full - >>> flow control off >>> >>> [ 462.760705] nixge 40000000.ethernet sfp0: Link is Down >>> >>> [ 463.880678] nixge 40000000.ethernet sfp0: Link is Up - 10Gbps/Full - >>> flow control off >>> >>> [ 466.120673] nixge 40000000.ethernet sfp0: Link is Down >>> >>> >>> uhd_usrp_probe: >>> >>> _____________________________________________________ >>> >>> / >>> >>> | Device: N300-Series Device >>> >>> | _____________________________________________________ >>> >>> | / >>> >>> | | Mboard: ni-n3xx-31E00AC >>> >>> | | dboard_0_pid: 338 >>> >>> | | dboard_0_serial: 31DB406 >>> >>> | | dboard_1_pid: 338 >>> >>> | | dboard_1_serial: 31DB407 >>> >>> | | eeprom_version: 3 >>> >>> | | fs_version: 20200914000806 >>> >>> | | mender_artifact: v4.0.0.0_n3xx >>> >>> | | mpm_sw_version: 4.0.0.0-g90ce6062 >>> >>> | | pid: 16962 >>> >>> | | product: n320 >>> >>> | | rev: 7 >>> >>> | | rpc_connection: local >>> >>> | | serial: 31E00AC >>> >>> | | type: n3xx >>> >>> | | MPM Version: 3.0 >>> >>> | | FPGA Version: 8.0 >>> >>> | | FPGA git hash: be53058.clean >>> >>> | | >>> >>> | | Time sources: internal, external, gpsdo, sfp0 >>> >>> | | Clock sources: external, internal, gpsdo >>> >>> | | Sensors: ref_locked, gps_locked, temp, fan, gps_gpgga, gps_sky, >>> gps_time, gps_tpv >>> >>> >>> Are there any configuration items needed to connect to the N321 through >>> the QSFP+ port. Note that I only see eth0, sfp0, sfp1, and int0 in >>> /etc/network/interfaces. >>> >>> >>> Thanks, >>> >>> Pat >>> >>> >>> _______________________________________________ >>> USRP-users mailing list >>> USRP-users@lists.ettus.com >>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>> >>>
_______________________________________________ USRP-users mailing list USRP-users@lists.ettus.com http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
