Wade,

Thank you for the links.  I was able to resolve the issue on my Ubuntu/X410 
system by setting the CPU governor to “performance” mode.

for ((i=0;i<$(nproc --all);i++)); do sudo cpufreq-set -c $i -r -g performance; 
done

I have a similar “underrun” issue on a CentOS 7.6.1810 (kernel 3.10.0-957) 
connected to a USRP X310 using UHD 4.6.0.  The network card is an Intel X710 
with 10Gbit ethernet connection to USRP.

On this system, I’ve disabled hyperthreading, cpu scaling, and virtualization 
via the BIOS.  I’ve confirmed that I can run the benchmark test with elevated 
thread priorities and nice.

I’ve configured the following network settings.

sysctl -w net.core.wmem_max=33554432
sysctl -w net.core.rmem_max=33554432
sysctl -w net.core.wmem_default=33554432

sysctl -w net.core.rmem_default=33554432
ethtool -G ens8f0np0 tx 4096 rx 4096

ethtool -A ens8f0np0 tx on

When I run the benchmark_rate test for 1 channel at 50MSps with priority=high, 
I get a consistent underrun at about 1Hz.  This is similar to the issue 
described in the 2nd link you provided.

Finally, I’ve confirmed that I can run the benchmark_rate test successfully (no 
underruns) with UHD version 3.14.1.

Is there an issue with a newer UHD (4.6.0) and an older OS (CentOS 7.6)?  Are 
there any optimizations that are perhaps different on the older CentOS-based 
system?

Jeremy

From: Wade Fife <wade.f...@ettus.com>
Sent: Tuesday, January 21, 2025 11:08 PM
To: Pallotta, Jeremy M (US) <jeremy.pallo...@baesystems.us>
Cc: usrp-users@lists.ettus.com
Subject: Re: [USRP-users] X410 benchmark_rate underrun at 50MSps

External Email Alert
This email has been sent from an account outside of the BAE Systems network.
Please treat the email with caution, especially if you are requested to click 
on a link, decrypt/open an attachment, or enable macros.  For further 
information on how to spot phishing, access “Cybersecurity OneSpace Page” and 
report phishing by clicking the button “Report Phishing” on the Outlook toolbar.

Hi Jeremy,

This is a good reference of things you can try:
https://kb.ettus.com/USRP_Host_Performance_Tuning_Tips_and_Tricks
Also, I wonder if RT_RUNTIME_SHARE could be causing problems? You can find some 
info here on how to enable it:
https://kb.ettus.com/Getting_Started_with_DPDK_and_UHD#Underruns_Every_Second_with_DPDK_.2B_Ubuntu

Another thing that comes to mind is enabling pause frames on the X410 as 
described here:
https://files.ettus.com/manual/page_transport.html#transport_udp_linux

Wade

On Tue, Jan 21, 2025 at 7:40 AM Pallotta, Jeremy M (US) 
<jeremy.pallo...@baesystems.us<mailto:jeremy.pallo...@baesystems.us>> wrote:

I am running into underflow issues when running the benchmark_rate test on a 
x410.  The command I am running is the following:



./benchmark_rate 
--args="addr=192.168.20.2,type=x4xx,send_frame_size=8000,recv_frame_size=8000" 
--tx_rate=50e6 --tx_channels=0 --tx_delay=1 --duration=60 --priority=high 
--ref=internal



I have a 100Gbit interface connected to the x410 using the below ethernet card.



ca:00.0 Ethernet controller: Mellanox Technologies MT2910 Family [ConnectX-7]



I'm running on Ubuntu 22.04.5 LTS with kernel 6.8.0-40-generic.



I've confirmed that the priority=high setting is working.  "top" shows the 
priorities of all the threads have been elevated.



I've also updated a number of network settings.



sysctl -w net.core.wmem_max=250000000
sysctl -w net.core.rmem_max=250000000
sysctl -w net.core.wmem_default=250000000
sysctl -w net.core.rmem_default=250000000
ethtool -G ens8f0np0 tx 8192 rx 8192



The application outputs the following:



[INFO] [UHD] linux; GNU C++ version 8.4.0; Boost_108300; 
UHD_4.6.0.HEAD-0-g50fa3baa
[00:00:00.000277] Creating the usrp device with: 
addr=192.168.20.2,type=x4xx,send_frame_size=8000,recv_frame_size=8000...
[INFO] [MPMD] Initializing 1 device(s) in parallel with args: 
mgmt_addr=192.168.20.2,type=x4xx,product=x410,serial=3289B1C,name=ni-x4xx-3289 
B1C,fpga=UC_200,claimed=False,addr=192.168.20.2,send_frame_size=8000,recv_frame_size=8000
[INFO] [MPM.PeriphManager] init() called with device args 
`fpga=UC_200,mgmt_addr=192.168.20.2,name=ni-x4xx-3289B1C,product=x410,recv_frame_s
 
ize=8000,send_frame_size=8000,clock_source=internal,time_source=internal,initializing=True'.
Using Device: Single USRP:
Device: X400-Series Device
Mboard 0: x410
RX Channel: 0
RX DSP: 0
RX Dboard: A
RX Subdev: 0
RX Channel: 1
RX DSP: 1
RX Dboard: A
RX Subdev: 1
RX Channel: 2
RX DSP: 2
RX Dboard: B
RX Subdev: 0
RX Channel: 3
RX DSP: 3
RX Dboard: B
RX Subdev: 1
TX Channel: 0
TX DSP: 0
TX Dboard: A
TX Subdev: 0
TX Channel: 1
TX DSP: 1
TX Dboard: A
TX Subdev: 1
TX Channel: 2
TX DSP: 2
TX Dboard: B
TX Subdev: 0
TX Channel: 3
TX DSP: 3
TX Dboard: B
TX Subdev: 1

[00:00:02.516914784] Setting device timestamp to 0...
[WARNING] [0/DUC#0] The requested interpolation is odd; the user should expect 
passband CIC rolloff.
Select an even interpolation to ensure that a halfband filter is enabled.

[WARNING] [MULTI_USRP] Could not set TX rate to 50.000 MHz. Actual rate is 
49.152 MHz
[WARNING] [0/DUC#0] The requested interpolation is odd; the user should expect 
passband CIC rolloff.
Select an even interpolation to ensure that a halfband filter is enabled.

[WARNING] [MULTI_USRP] Could not set TX rate to 50.000 MHz. Actual rate is 
49.152 MHz
[WARNING] [0/DUC#1] The requested interpolation is odd; the user should expect 
passband CIC rolloff.
Select an even interpolation to ensure that a halfband filter is enabled.

[WARNING] [MULTI_USRP] Could not set TX rate to 50.000 MHz. Actual rate is 
49.152 MHz
[WARNING] [0/DUC#1] The requested interpolation is odd; the user should expect 
passband CIC rolloff.
Select an even interpolation to ensure that a halfband filter is enabled.

[WARNING] [MULTI_USRP] Could not set TX rate to 50.000 MHz. Actual rate is 
49.152 MHz
[WARNING] [0/Radio#0] Attempting to set tick rate to 0. Skipping.
Setting TX spp to 1992
[00:00:02.522008533] Testing transmit rate 49.152000 Msps on 1 channels
UUUUUUUUUUUUUUUUUUUUUUU[00:01:03.524365256] Benchmark complete.

Benchmark rate summary:
Num received samples: 0
Num dropped samples: 0
Num overruns detected: 0
Num transmitted samples: 2948209800
Num sequence errors (Tx): 0
Num sequence errors (Rx): 0
Num underruns detected: 23
Num late commands: 0
Num timeouts (Tx): 0
Num timeouts (Rx): 0

Done!



Does anyone have any other suggestions for what could be tried to avoid the 
underruns?  With a 100Gbit/s NIC, I should have plenty of network throughput.

_______________________________________________
USRP-users mailing list -- 
usrp-users@lists.ettus.com<mailto:usrp-users@lists.ettus.com>
To unsubscribe send an email to 
usrp-users-le...@lists.ettus.com<mailto:usrp-users-le...@lists.ettus.com>
_______________________________________________
USRP-users mailing list -- usrp-users@lists.ettus.com
To unsubscribe send an email to usrp-users-le...@lists.ettus.com

Reply via email to