On 28/04/2025 19:08, Nikos Balkanas wrote:
On Tue, Apr 29, 2025 at 1:31 AM Marcus D. Leech <patchvonbr...@gmail.com> wrote:On 28/04/2025 18:04, Nikos Balkanas wrote:Hi Marcus, spb(=samples/buffer) is not what I hoped for. It is just another way to set the stream_cmd.num_samples in the streamer. I already use there MAXSMPS. Besides benchmark_rate reports maxsmps 1996 like me:( That part is only controlled by the MTU. Seems it is the same for everyone, and that we are using the generic chdr_sc16_to_xx instead the guts (sse2) conversion:( Unless they use the multi_usrp class and set spp on their own. I think that MAXSMPS used to be 1996, before someone changed it to 19960. BR Nikos BR NikosWell, regardless of all of this, there's no way to pack more than a certain number of samples into an MTU frame. If your high-level needs are to process "things" in larger chunks, you need to layer something on top of what UHD does, since it is, at the end of the day, a hardware driver library. It's not an application layer programming framework.Not really. I just need for FFT 1024 samples at the moment. My concern is more about the sse2 code not used:(What about packet fragmentation? NICs can handle those...
I think that by default UDP on most systems these days sets the "DF" bit -- so that higher-layer protocols like UDP restrict themselves to non-fragmented packets at that level. That's because in the larger internet, you can't rely on in-order fragment handling, and re-ordering is a HUGE performance loss.
Anyway, for a 1024-sample FFT, you have what you need.
On Mon, Apr 28, 2025 at 5:22 PM Nikos Balkanas <nbalka...@gmail.com> wrote: Thx Marcus for the clarifications, On Mon, Apr 28, 2025 at 4:37 PM Marcus D. Leech <patchvonbr...@gmail.com> wrote: On 28/04/2025 05:33, Nikos Balkanas wrote:Compiled uhd 4.6.0 in debug mode. From the output I get: [DEBUG] [0/Radio#0] spp(= samples per package) value 2032 exceeds MTU of 8000! Coercing to 1996 Mon Apr 28 09:57:02 2025 [00] [*] scanner.l:1443:main Incorrect maxsamples (1996). Expected 19960. Mon Apr 28 09:57:02 2025 [00] [+] Max samples/buffer[0]: 1996 1) Line mtu is 9000 not 8000 2) 2032 is not larger than 8000 <= Bug? 3) seems that spp is setting my maxsmpsThat's number of *SAMPLES*. Samples are 4 bytes total. Aaaah. I'll have to check the SPB option. Otherwise an 80K MTU is unreasonable:) This also means that 1 sample = 1 real + 1 imag = 32 bits with sc16 encoding . A lot of network hardware, particularly 1Gbit hardware doesn't *ACTUALLY* support an MTU of more than 8000, and I think UHD uses PMTU discovery. I found that with RealTek NICs, even when you set the MTU to 9000, it actually only supports 8000. Same case with my Mellanox NIC. But 8000 is close enough:)Of the examples I tried the rx_samples_c. It is the same case like mine: single usrp. We use the same commands and we are getting the same output:( 1996 maxsmpls. The error text and code are from host/lib/rfnoc/radio_control_impl.cpp: 199 I would rather not touch it. I don't know the uhd architecture and especially the rfnoc/uhd interface. Besides I am a c programmer, not c++:( multi_usrp class has a set_rx_spp function, but it is not for me:(You can look at the benchmark_rate example to see how to set a samples-per-buffer other than the default, which is based on the MTU. It uses an "SPB" command-line parameter. Thx, I will check it out, when I get back to ubuntu.. Now I am in windows:( benchmark_rate uses the multi-usrp class.HTH Nikos On Mon, Apr 28, 2025 at 6:02 AM Nikos Balkanas <nbalka...@gmail.com> wrote: Point taken:) I'm proposing smt different: pchar +ICMP are just to test line and connectors. First step. Not to bench USRP. benchmark_rate is to bench/stress usrp. These 2 are independent, and complementary. Pchar is telling me nothing more than my fiber cable and connectors are good. It saved me a trip tomorrow to my local PC store, where fiber cable and connectors are ~7 E each. benchmark_rate on the other hand is quite interesting. It points to software,and particularly my uhd_init() Just found and downloaded the sources to uhd 4.6.0 from Ubuntu Launchpad. Now I can go through the source of the example you told me:) Ettus used to keep an archive of old uhd sources around. Seems it's gone:( Open source means, among others, free to choose the source version that you need... Having the latest source in Github is only partly open source. During development we need to freeze updates. When in 5 years we go into production we can't find the old sources anymore:( If a customer discovers a bug, not in our code, but in one of the libraries that we use, what are we gonna do? BR Nikos On Mon, Apr 28, 2025 at 5:01 AM Marcus D. Leech <patchvonbr...@gmail.com> wrote: On 27/04/2025 21:58, Nikos Balkanas wrote:My bad: throughput of 5.619 Kb/s requesting ICMP replies, +> throughput of 5,619 Kb/s requesting ICMP replies Local thousand separator is ".", whereas in the US is ",":(It is STILL the case that the ICMP machinery in these radios is ABSOLUTELY NOT on the fast-path inside the hardware. The only way to get a good feel for how much sample bandwidth they can process is with examples like "benchmark_rate".On Mon, Apr 28, 2025 at 12:37 AM Nikos Balkanas <nbalka...@gmail.com> wrote: Hi Marcus, You were right. No need to change NIC:) This is not a software issue. uhd_rx_streamer_max_num_samps runs right after uhd initialization before any other code had the chance to run. Link capacity doesn't seem to be the issue either... Running pchar on the link, descendant of pathchar, reports a throughput of 5.619 Kb/s requesting ICMP replies, to varying packet sizes (32->9000 (MTU), incr by 32). sudo pchar -m 9000 -p ipv4icmp usrp https://www.kitchenlab.org/www/bmah/Software/pchar/ It corresponds to 351.218.019 16-bit samples or 175,609.044 32-bit samples, if each sample is 32-bit(real + imag) Seems that uhd is not running at link capacity but is doing smt else. I will have to download and check with the sources... The package version for Ubuntu 24.04 is uhd 4.6.0. Where can I download the sources for uhd 4.6.0? BR Nikos On Sat, Apr 26, 2025 at 7:02 AM Nikos Balkanas <nbalka...@gmail.com> wrote: Thanks for your time. I will check out the example. This is not a buffer problem. I just need 1024 Samples (real+imaginary) for FFT... I should be able to get them in a single pass. You saw my code, not a smoking gun there. This is probably is a physical problem. Cable is an SFP fiber dedicated line. Cannot go bad. Maybe the connections are not sitting right :(... BR Nikos On Sat, Apr 26, 2025 at 6:45 AM Marcus D. Leech <patchvonbr...@gmail.com> wrote: On 25/04/2025 23:33, Nikos Balkanas wrote:Actually MTU is 9000. This is one of the recommendations... I tried it with MTU 1500. It was worse:( maxsamples dropped to 364...Right, 9000, rather than 8000. Upgrading to 10Gbit wont' give you larger MTU. What you're trying to do, I think, is to solve a buffer-management problem in your *application* at entirely the wrong level in the stack. It is EXCEEDINGLY COMMON for hardware drivers to only be able to deliver in chunks that may not be perfectly adapted to the requirements of your application. So, a common programming pattern is to deal with this in your application. You should probably look at example code like rx_samples_to_file[INFO] [UHD] linux; GNU C++ version 13.2.0; Boost_108300; UHD_4.6.0.0+ds1-5.1ubuntu0.24.04.1 [INFO] [X300] X300 initialization sequence... [INFO] [X300] Maximum frame size: 1472 bytes. [WARNING] [X300] For the 192.168.40.2 connection, UHD recommends a send frame size of at least 8000 for best performance, but your configuration will only allow 1472.This may negatively impact your maximum achievable sample rate. Check the MTU on the interface and/or the send_frame_size argument. [WARNING] [X300] For the 192.168.40.2 connection, UHD recommends a receive frame size of at least 8000 for best performance, but your configuration will only allow 1472.This may negatively impact your maximum achievable sample rate. Check the MTU on the interface and/or the recv_frame_size argument. [INFO] [GPS] No GPSDO found [INFO] [X300] Radio 1x clock: 200 MHz [WARNING] [UDP] The send buffer could not be resized sufficiently. Target sock buff size: 24912805 bytes. Actual sock buff size: 1048576 bytes. See the transport application notes on buffer resizing. Please run: sudo sysctl -w net.core.wmem_max=24912805 Sat Apr 26 06:30:34 2025 [00] [+] Created USRP with args Sat Apr 26 06:30:34 2025 [00] [+] Master clock is at 200 Mhz Sat Apr 26 06:30:34 2025 [00] [+] Tuner[0] gain set to 30 (30) dB [WARNING] [UDP] The send buffer could not be resized sufficiently. Target sock buff size: 24912805 bytes. Actual sock buff size: 1048576 bytes. See the transport application notes on buffer resizing. Please run: sudo sysctl -w net.core.wmem_max=24912805 Sat Apr 26 06:30:34 2025 [00] [*] scanner.l:1446:main Incorrect maxsamples (364). Expected 19960. Sat Apr 26 06:30:34 2025 [00] [+] Max samples/buffer[0]: 364 [WARNING] [0/Radio#0] Ignoring stream command for finite acquisition of zero sam Nikos On Sat, Apr 26, 2025 at 5:46 AM Marcus D. Leech <patchvonbr...@gmail.com> wrote: On 25/04/2025 22:26, Nikos Balkanas wrote:Thanks Marcus, for your fast reply. On Sat, Apr 26, 2025 at 4:08 AM Marcus D. Leech <patchvonbr...@gmail.com> wrote: On 25/04/2025 20:50, Nikos Balkanas wrote:Hello, I need to buy a new NIC. What would you suggest? The one I use is an old Mellanox 10 Gbs, before the Connect-4 series. It can only do 1996 S/s, need 19960 (10x more) to work with latest uhd. Using Ubuntu 24.04 and uhd 4.6.0So, 1.996ksps vs 19.960ksps? You hardly need a 10Gbit link to support that. So, perhaps something is being lost here in your requirements? True. Can't explain it in terms of bandwidth. 16 * 1996 = 31.936 Kbps, 16 * 19960 = 319.360 Kbps well short of a 10 Gbps line:( Does a complex pair count as 1 sample, or 2? I have followed all the instructions in https://kb.ettus.com/USRP_Host_Performance_Tuning_Tips_and_Tricks, Even installed the DPDK drivers. My Mellanox is too old to use their OFED drivers:(On a related question. it seems that the streamer doesn't crash anymore when receiving less than MAXSPS samples, instead it reads 0:( This was due to the sse2 code not aligned in convert. I change my stream args to cpu_format=sc16, otw=sc16, so no conversion required. Streamer still doesn't read anything. Is there a reason for it?You'd need to share more of your code. This should just work as far as I can tell. Thanks. these are just the usrp code: int main() { uhd_stream_args_t stream_args = { .cpu_format = "sc16", .otw_format = "sc16", .args = "", .n_channels = 1, .channel_list = &channel }; ..uhd_stream_cmd_t stream_cmd = { .stream_mode = UHD_STREAM_MODE_NUM_SAMPS_AND_DONE, .stream_now = true }; if (uhd_init(0, 0, gain)) do_exit(20); if ((err = uhd_usrp_get_rx_stream(dev[0], &stream_args, rx_streamer[0]))) { uhd_get_last_error(errmsg, 127); error(log, "Failed to get streamer[0] (%d). %s.\n", 0, FL, LN, FN, err, errmsg); uhd_rx_streamer_free(&rx_streamer[0]); rx_streamer[0] = NULL; uhd_rx_metadata_free(&md[0]); md[0] = NULL; do_exit(30); } if ((err = uhd_rx_streamer_max_num_samps(rx_streamer[0], &maxsamps))) { uhd_get_last_error(errmsg, 127); error(log, "Failed to get max samples/buffer[0] (%d). %s.\n", 0, FL, LN, FN, err, ..errmsg); do_exit(35); } if (maxsamps != MAXSMPS) warn(log, "Incorrect maxsamples (%ld). Expected %d.\n", 0, FL, LN, FN, maxsamps, MAXSMPS); info(log, "Max samples/buffer[0]: %ld\n", 0, maxsamps); if ((err = uhd_rx_streamer_issue_stream_cmd(rx_streamer[0], &stream_cmd))) { uhd_get_last_error(errmsg, 127); error(log, "Failed to start streamer[0] (%d). %s.\n", 0, FL, LN, FN, err, errmsg); do_exit(40); } [...] do_exit(0) } bool uhd_init(size_t channel, double srate, double gain) { double tmp; uhd_rx_metadata_error_code_t err; if ((err = uhd_set_thread_priority(uhd_default_thread_priority, true))) warn(log, "Unable to set main thread priority (%d). %s.\n", 0, FL, LN, FN, err, uhdError(err)); /* Create USRP */ f ((err = uhd_usrp_make(&dev[channel], "type=x300"))) { error(log, "Failed to create USRP (%d). %s.\n", 0, FL, LN, FN, err, uhdError(err)); dev[channel] = NULL; return(FAIL); } info(stderr, "Created USRP with args\n", 0); /* Create RX streamer */ if ((err = uhd_rx_streamer_make(&rx_streamer[channel]))) { error(log, "Failed to create rx_streamer[%d] (%d). %s.\n", 0, FL, LN, FN, channel, err, uhdError(err)); return(FAIL); } /* Create RX metadata */ if ((err = uhd_rx_metadata_make(&md[channel]))) { error(log, "Failed to create md[%d] (%d). %s.\n", 0, FL, LN, FN, channel, err, uhdError(err)); return(FAIL); } /* Get master clock rate */ if ((err = uhd_usrp_get_master_clock_rate(dev[channel], 0, &tmp))) { error(log, "Failed to set master clock to %.0lf Mhz (%d). %s.\n", 0, FL, LN, FN, tmp/1000000, err, uhdError(err)); return(FAIL); } info(stderr, "Master clock is at %.0lf Mhz\n", 0, tmp/1000000); /* Set the sample rate */ if (srate && !uhd_set_rx_rate_check(channel, srate)) return(FAIL); /* Set the tuner gain SBX-120 is 0-31.5 in .5 db steps */ if ((err = uhd_usrp_set_rx_gain(dev[channel], gain, channel, ""))) { error(log, "Failed to set tuner[%d] gain to %.0lf db (%d). %s.\n", 0, FL, LN, FN, channel, gain, err, uhdError(err)); return(FAIL); } if (!(err = uhd_usrp_get_rx_gain(dev[channel], channel, "", &tmp))) info(log, "Tuner[%d] gain set to %.0lf (%.0lf) dB\n", 0, channel, tmp,gain); ./* Set channel bw to conserve tuner resources. Not needed, set by srate */ uhd_usrp_set_rx_bandwidth(dev[channel], srate, channel); ./* Disable subtracting constant averaged background. Signal looks cleaner */ if ((err = uhd_usrp_set_rx_dc_offset_enabled(dev[channel], false, channel))) { warn(log, "Failed to disable FPGA DC offset on channel %d(%d). %s.\n", 0, FL, LN, FN, channel, err, uhdError(err)); } info(stderr, "Disabled FPGA DC offset on channel %d\n", 0, channel); return(SUCCESS); } This is the generated output: [INFO] [UHD] linux; GNU C++ version 13.2.0; Boost_108300; UHD_4.6.0.0+ds1-5.1ubuntu0.24.04.1 [INFO] [X300] X300 initialization sequence... [INFO] [X300] Maximum frame size: 8000 bytes. [INFO] [X300] Radio 1x clock: 200 MHz Sat Apr 26 03:33:48 2025 [00] [+] Created USRP with args Sat Apr 26 03:33:48 2025 [00] [+] Master clock is at 200 Mhz Sat Apr 26 03:33:48 2025 [00] [+] Tuner[0] gain set to 30 (30) dB Sat Apr 26 03:33:48 2025 [00] [*] scanner.l:1446:main Incorrect maxsamples (1996). Expected 19960. Sat Apr 26 03:33:48 2025 [00] [+] Max samples/buffer[0]: 1996 [WARNING] [0/Radio#0] Ignoring stream command for finite acquisition of zero samples I hope this reads OK. Maybe next time I should attach the code:)TIA Nikos _______________________________________________ USRP-users mailing list --usrp-users@lists.ettus.com To unsubscribe send an email tousrp-users-le...@lists.ettus.comI believe that max number of samples-per-buffer is limited by MTU size. Which is typically around 8000 or so for "jumbo frames".
_______________________________________________ USRP-users mailing list -- usrp-users@lists.ettus.com To unsubscribe send an email to usrp-users-le...@lists.ettus.com
