On Tue, Apr 29, 2025 at 1:31 AM Marcus D. Leech <patchvonbr...@gmail.com> wrote:
> On 28/04/2025 18:04, Nikos Balkanas wrote: > > Hi Marcus, > > spb(=samples/buffer) is not what I hoped for. It is just another way to > set the > stream_cmd.num_samples in the streamer. I already use there MAXSMPS. > Besides benchmark_rate reports maxsmps 1996 like me:( > That part is only controlled by the MTU. Seems it is the same > for everyone, and that we are using the generic chdr_sc16_to_xx > instead the guts (sse2) conversion:( Unless they use the multi_usrp > class and set spp on their own. > I think that MAXSMPS used to be 1996, before someone changed it to > 19960. > > BR > Nikos > > BR > Nikos > > Well, regardless of all of this, there's no way to pack more than a > certain number of samples into an MTU frame. > > If your high-level needs are to process "things" in larger chunks, you > need to layer something on top of what UHD > does, since it is, at the end of the day, a hardware driver library. > It's not an application layer programming framework. > > Not really. I just need for FFT 1024 samples at the moment. My concern is more about the sse2 code not used:( What about packet fragmentation? NICs can handle those... > > On Mon, Apr 28, 2025 at 5:22 PM Nikos Balkanas <nbalka...@gmail.com> > wrote: > >> Thx Marcus for the clarifications, >> >> On Mon, Apr 28, 2025 at 4:37 PM Marcus D. Leech <patchvonbr...@gmail.com> >> wrote: >> >>> On 28/04/2025 05:33, Nikos Balkanas wrote: >>> >>> Compiled uhd 4.6.0 in debug mode. >>> From the output I get: >>> >>> [DEBUG] [0/Radio#0] spp(= samples per package) value 2032 exceeds MTU of >>> 8000! Coercing to 1996 >>> Mon Apr 28 09:57:02 2025 [00] [*] scanner.l:1443:main Incorrect >>> maxsamples (1996). Expected 19960. >>> Mon Apr 28 09:57:02 2025 [00] [+] Max samples/buffer[0]: 1996 >>> 1) Line mtu is 9000 not 8000 >>> 2) 2032 is not larger than 8000 <= Bug? >>> 3) seems that spp is setting my maxsmps >>> >>> That's number of *SAMPLES*. Samples are 4 bytes total. >>> >> >> Aaaah. I'll have to check the SPB option. Otherwise an 80K MTU is >> unreasonable:) >> This also means that 1 sample = 1 real + 1 imag = 32 bits with sc16 >> encoding >> >> . >>> >> >> A lot of network hardware, particularly 1Gbit hardware doesn't *ACTUALLY* >>> support an MTU of more than 8000, and I think >>> UHD uses PMTU discovery. I found that with RealTek NICs, even when >>> you set the MTU to 9000, it actually only supports >>> 8000. >>> >>> Same case with my Mellanox NIC. But 8000 is close enough:) >> >>> >>> >>> Of the examples I tried the rx_samples_c. It is the same case like mine: >>> single usrp. We use the same commands >>> and we are getting the same output:( 1996 maxsmpls. >>> The error text and code are from host/lib/rfnoc/radio_control_impl.cpp: >>> 199 >>> I would rather not touch it. I don't know the uhd architecture and >>> especially the rfnoc/uhd interface. >>> Besides I am a c programmer, not c++:( >>> multi_usrp class has a set_rx_spp function, but it is not for me:( >>> >>> You can look at the benchmark_rate example to see how to set a >>> samples-per-buffer other than the default, which is >>> based on the MTU. It uses an "SPB" command-line parameter. >>> >>> Thx, I will check it out, when I get back to ubuntu.. Now I am in >> windows:( >> benchmark_rate uses the multi-usrp class. >> >>> >>> HTH >>> Nikos >>> >>> On Mon, Apr 28, 2025 at 6:02 AM Nikos Balkanas <nbalka...@gmail.com> >>> wrote: >>> >>>> Point taken:) I'm proposing smt different: >>>> pchar +ICMP are just to test line and connectors. First step. Not to >>>> bench USRP. >>>> benchmark_rate is to bench/stress usrp. >>>> These 2 are independent, and complementary. >>>> Pchar is telling me nothing more than my fiber cable and connectors are >>>> good. >>>> It saved me a trip tomorrow to my local PC store, where fiber cable and >>>> connectors are ~7 E each. >>>> benchmark_rate on the other hand is quite interesting. >>>> It points to software,and particularly my uhd_init() >>>> >>>> Just found and downloaded the sources to uhd 4.6.0 from Ubuntu >>>> Launchpad. >>>> Now I can go through the source of the example you told me:) >>>> Ettus used to keep an archive of old uhd sources around. Seems it's >>>> gone:( >>>> Open source means, among others, free to choose the source version that >>>> you need... >>>> Having the latest source in Github is only partly open source. >>>> During development we need to freeze updates. When in 5 years we go >>>> into production we can't find the old sources anymore:( >>>> If a customer discovers a bug, not in our code, but in one of the >>>> libraries that >>>> we use, what are we gonna do? >>>> >>>> BR >>>> Nikos >>>> >>>> >>>> >>>> On Mon, Apr 28, 2025 at 5:01 AM Marcus D. Leech < >>>> patchvonbr...@gmail.com> wrote: >>>> >>>>> On 27/04/2025 21:58, Nikos Balkanas wrote: >>>>> >>>>> My bad: >>>>> >>>>> throughput of 5.619 Kb/s requesting ICMP replies, +> throughput of >>>>> 5,619 Kb/s requesting ICMP replies >>>>> Local thousand separator is ".", whereas in the US is ",":( >>>>> >>>>> It is STILL the case that the ICMP machinery in these radios is >>>>> ABSOLUTELY NOT on the fast-path inside >>>>> the hardware. The only way to get a good feel for how much sample >>>>> bandwidth they can process is >>>>> with examples like "benchmark_rate". >>>>> >>>>> >>>>> >>>>> On Mon, Apr 28, 2025 at 12:37 AM Nikos Balkanas <nbalka...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hi Marcus, >>>>>> >>>>>> You were right. No need to change NIC:) >>>>>> This is not a software issue. uhd_rx_streamer_max_num_samps runs >>>>>> right after uhd initialization before >>>>>> any other code had the chance to run. >>>>>> Link capacity doesn't seem to be the issue either... >>>>>> Running pchar on the link, descendant of pathchar, reports a >>>>>> throughput of 5.619 Kb/s requesting ICMP replies, >>>>>> to varying packet sizes (32->9000 (MTU), incr by 32). >>>>>> sudo pchar -m 9000 -p ipv4icmp usrp >>>>>> https://www.kitchenlab.org/www/bmah/Software/pchar/ >>>>>> >>>>>> It corresponds to 351.218.019 16-bit samples or 175,609.044 32-bit >>>>>> samples, if each sample is 32-bit(real + imag) >>>>>> Seems that uhd is not running at link capacity but is doing smt else. >>>>>> I will have to download and check with the sources... >>>>>> The package version for Ubuntu 24.04 is uhd 4.6.0. >>>>>> Where can I download the sources for uhd 4.6.0? >>>>>> >>>>>> BR >>>>>> Nikos >>>>>> >>>>>> On Sat, Apr 26, 2025 at 7:02 AM Nikos Balkanas <nbalka...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Thanks for your time. >>>>>>> I will check out the example. >>>>>>> This is not a buffer problem. I just need 1024 Samples >>>>>>> (real+imaginary) for FFT... >>>>>>> I should be able to get them in a single pass. >>>>>>> You saw my code, not a smoking gun there. >>>>>>> >>>>>>> This is probably is a physical problem. >>>>>>> Cable is an SFP fiber dedicated line. Cannot go bad. >>>>>>> Maybe the connections are not sitting right :(... >>>>>>> >>>>>>> BR >>>>>>> Nikos >>>>>>> >>>>>>> On Sat, Apr 26, 2025 at 6:45 AM Marcus D. Leech < >>>>>>> patchvonbr...@gmail.com> wrote: >>>>>>> >>>>>>>> On 25/04/2025 23:33, Nikos Balkanas wrote: >>>>>>>> >>>>>>>> Actually MTU is 9000. This is one of the recommendations... >>>>>>>> I tried it with MTU 1500. It was worse:( >>>>>>>> maxsamples dropped to 364... >>>>>>>> >>>>>>>> Right, 9000, rather than 8000. >>>>>>>> >>>>>>>> Upgrading to 10Gbit wont' give you larger MTU. >>>>>>>> >>>>>>>> What you're trying to do, I think, is to solve a buffer-management >>>>>>>> problem in your *application* at entirely the wrong >>>>>>>> level in the stack. >>>>>>>> >>>>>>>> It is EXCEEDINGLY COMMON for hardware drivers to only be able to >>>>>>>> deliver in chunks that may not be perfectly adapted to >>>>>>>> the requirements of your application. So, a common programming >>>>>>>> pattern is to deal with this in your application. >>>>>>>> >>>>>>>> You should probably look at example code like rx_samples_to_file >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> [INFO] [UHD] linux; GNU C++ version 13.2.0; Boost_108300; >>>>>>>> UHD_4.6.0.0+ds1-5.1ubuntu0.24.04.1 >>>>>>>> [INFO] [X300] X300 initialization sequence... >>>>>>>> [INFO] [X300] Maximum frame size: 1472 bytes. >>>>>>>> [WARNING] [X300] For the 192.168.40.2 connection, UHD recommends a >>>>>>>> send frame size of at least 8000 for best >>>>>>>> performance, but your configuration will only allow 1472.This may >>>>>>>> negatively impact your maximum achievable sample rate. >>>>>>>> Check the MTU on the interface and/or the send_frame_size argument. >>>>>>>> [WARNING] [X300] For the 192.168.40.2 connection, UHD recommends a >>>>>>>> receive frame size of at least 8000 for best >>>>>>>> performance, but your configuration will only allow 1472.This may >>>>>>>> negatively impact your maximum achievable sample rate. >>>>>>>> Check the MTU on the interface and/or the recv_frame_size argument. >>>>>>>> [INFO] [GPS] No GPSDO found >>>>>>>> [INFO] [X300] Radio 1x clock: 200 MHz >>>>>>>> [WARNING] [UDP] The send buffer could not be resized sufficiently. >>>>>>>> Target sock buff size: 24912805 bytes. >>>>>>>> Actual sock buff size: 1048576 bytes. >>>>>>>> See the transport application notes on buffer resizing. >>>>>>>> Please run: sudo sysctl -w net.core.wmem_max=24912805 >>>>>>>> Sat Apr 26 06:30:34 2025 [00] [+] Created USRP with args >>>>>>>> Sat Apr 26 06:30:34 2025 [00] [+] Master clock is at 200 Mhz >>>>>>>> Sat Apr 26 06:30:34 2025 [00] [+] Tuner[0] gain set to 30 (30) dB >>>>>>>> [WARNING] [UDP] The send buffer could not be resized sufficiently. >>>>>>>> Target sock buff size: 24912805 bytes. >>>>>>>> Actual sock buff size: 1048576 bytes. >>>>>>>> See the transport application notes on buffer resizing. >>>>>>>> Please run: sudo sysctl -w net.core.wmem_max=24912805 >>>>>>>> Sat Apr 26 06:30:34 2025 [00] [*] scanner.l:1446:main Incorrect >>>>>>>> maxsamples (364). Expected 19960. >>>>>>>> Sat Apr 26 06:30:34 2025 [00] [+] Max samples/buffer[0]: 364 >>>>>>>> [WARNING] [0/Radio#0] Ignoring stream command for finite >>>>>>>> acquisition of zero sam >>>>>>>> >>>>>>>> Nikos >>>>>>>> >>>>>>>> On Sat, Apr 26, 2025 at 5:46 AM Marcus D. Leech < >>>>>>>> patchvonbr...@gmail.com> wrote: >>>>>>>> >>>>>>>>> On 25/04/2025 22:26, Nikos Balkanas wrote: >>>>>>>>> >>>>>>>>> Thanks Marcus, >>>>>>>>> >>>>>>>>> for your fast reply. >>>>>>>>> >>>>>>>>> On Sat, Apr 26, 2025 at 4:08 AM Marcus D. Leech < >>>>>>>>> patchvonbr...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> On 25/04/2025 20:50, Nikos Balkanas wrote: >>>>>>>>>> >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> I need to buy a new NIC. What would you suggest? >>>>>>>>>> The one I use is an old Mellanox 10 Gbs, before the Connect-4 >>>>>>>>>> series. >>>>>>>>>> It can only do 1996 S/s, need 19960 (10x more) to work with >>>>>>>>>> latest uhd. >>>>>>>>>> Using Ubuntu 24.04 and uhd 4.6.0 >>>>>>>>>> >>>>>>>>>> So, 1.996ksps vs 19.960ksps? You hardly need a 10Gbit link to >>>>>>>>>> support that. So, perhaps something >>>>>>>>>> is being lost here in your requirements? >>>>>>>>>> >>>>>>>>> >>>>>>>>> True. Can't explain it in terms of bandwidth. 16 * 1996 = 31.936 >>>>>>>>> Kbps, 16 * 19960 = 319.360 Kbps well short of a 10 Gbps line:( >>>>>>>>> Does a complex pair count as 1 sample, or 2? >>>>>>>>> I have followed all the instructions in >>>>>>>>> https://kb.ettus.com/USRP_Host_Performance_Tuning_Tips_and_Tricks, >>>>>>>>> Even installed the DPDK drivers. My Mellanox is too old to use >>>>>>>>> their OFED drivers:( >>>>>>>>> >>>>>>>>>> >>>>>>>>>> On a related question. it seems that the streamer doesn't crash >>>>>>>>>> anymore >>>>>>>>>> when receiving less than MAXSPS samples, instead it reads 0:( >>>>>>>>>> This was due to the sse2 code not aligned in convert. >>>>>>>>>> I change my stream args to cpu_format=sc16, otw=sc16, so no >>>>>>>>>> conversion required. >>>>>>>>>> Streamer still doesn't read anything. Is there a reason for it? >>>>>>>>>> >>>>>>>>>> You'd need to share more of your code. This should just work as >>>>>>>>>> far as I can tell. >>>>>>>>>> >>>>>>>>>> Thanks. these are just the usrp code: >>>>>>>>> >>>>>>>>> int main() >>>>>>>>> { >>>>>>>>> uhd_stream_args_t stream_args = >>>>>>>>> { >>>>>>>>> >>>>>>>>> .cpu_format = "sc16", >>>>>>>>> >>>>>>>>> .otw_format = "sc16", >>>>>>>>> >>>>>>>>> .args = "", >>>>>>>>> >>>>>>>>> .n_channels = 1, >>>>>>>>> >>>>>>>>> .channel_list = &channel >>>>>>>>> }; >>>>>>>>> ..uhd_stream_cmd_t stream_cmd = >>>>>>>>> { >>>>>>>>> >>>>>>>>> .stream_mode = UHD_STREAM_MODE_NUM_SAMPS_AND_DONE, >>>>>>>>> >>>>>>>>> .stream_now = true >>>>>>>>> }; >>>>>>>>> >>>>>>>>> if (uhd_init(0, 0, gain)) do_exit(20); >>>>>>>>>> if ((err = uhd_usrp_get_rx_stream(dev[0], &stream_args, >>>>>>>>>> rx_streamer[0]))) >>>>>>>>>> { >>>>>>>>>> uhd_get_last_error(errmsg, 127); >>>>>>>>>> error(log, "Failed to get streamer[0] (%d). %s.\n", 0, >>>>>>>>>> FL, LN, FN, err, errmsg); >>>>>>>>>> uhd_rx_streamer_free(&rx_streamer[0]); >>>>>>>>>> rx_streamer[0] = NULL; >>>>>>>>>> uhd_rx_metadata_free(&md[0]); >>>>>>>>>> md[0] = NULL; >>>>>>>>>> do_exit(30); >>>>>>>>>> } >>>>>>>>>> if ((err = uhd_rx_streamer_max_num_samps(rx_streamer[0], >>>>>>>>>> &maxsamps))) >>>>>>>>>> >>>>>>>>> { >>>>>>>>>> uhd_get_last_error(errmsg, 127); >>>>>>>>>> error(log, "Failed to get max samples/buffer[0] (%d). >>>>>>>>>> %s.\n", 0, FL, LN, FN, err, >>>>>>>>>> ..errmsg); >>>>>>>>>> do_exit(35); >>>>>>>>>> } >>>>>>>>>> if (maxsamps != MAXSMPS) >>>>>>>>>> warn(log, "Incorrect maxsamples (%ld). Expected %d.\n", >>>>>>>>>> 0, FL, LN, FN, maxsamps, >>>>>>>>>> MAXSMPS); >>>>>>>>>> info(log, "Max samples/buffer[0]: %ld\n", 0, maxsamps); >>>>>>>>>> >>>>>>>>>> if ((err = uhd_rx_streamer_issue_stream_cmd(rx_streamer[0], >>>>>>>>>> &stream_cmd))) >>>>>>>>>> >>>>>>>>> { >>>>>>>>>> uhd_get_last_error(errmsg, 127); >>>>>>>>>> error(log, "Failed to start streamer[0] (%d). %s.\n", 0, >>>>>>>>>> FL, LN, FN, err, errmsg); >>>>>>>>>> do_exit(40); >>>>>>>>>> } >>>>>>>>>> >>>>>>>>> [...] >>>>>>>>> do_exit(0) >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>>> bool uhd_init(size_t channel, double srate, double gain) >>>>>>>>>> { >>>>>>>>>> double tmp; >>>>>>>>>> uhd_rx_metadata_error_code_t err; >>>>>>>>>> >>>>>>>>>> if ((err = >>>>>>>>>> uhd_set_thread_priority(uhd_default_thread_priority, true))) >>>>>>>>>> warn(log, "Unable to set main thread priority (%d). >>>>>>>>>> %s.\n", 0, FL, LN, FN, >>>>>>>>>> err, uhdError(err)); >>>>>>>>>> /* Create USRP */ >>>>>>>>>> f ((err = uhd_usrp_make(&dev[channel], "type=x300"))) >>>>>>>>>> { >>>>>>>>>> error(log, "Failed to create USRP (%d). %s.\n", 0, FL, >>>>>>>>>> LN, FN, err, >>>>>>>>>> uhdError(err)); >>>>>>>>>> dev[channel] = NULL; >>>>>>>>>> return(FAIL); >>>>>>>>>> >>>>>>>>> } >>>>>>>>>> info(stderr, "Created USRP with args\n", 0); >>>>>>>>>> >>>>>>>>>> /* Create RX streamer */ >>>>>>>>>> if ((err = uhd_rx_streamer_make(&rx_streamer[channel]))) >>>>>>>>>> { >>>>>>>>>> error(log, "Failed to create rx_streamer[%d] (%d). >>>>>>>>>> %s.\n", 0, FL, LN, FN, >>>>>>>>>> channel, err, uhdError(err)); >>>>>>>>>> return(FAIL); >>>>>>>>>> } >>>>>>>>>> /* Create RX metadata */ >>>>>>>>>> if ((err = uhd_rx_metadata_make(&md[channel]))) >>>>>>>>>> { >>>>>>>>>> error(log, "Failed to create md[%d] (%d). %s.\n", 0, FL, >>>>>>>>>> LN, FN, channel, >>>>>>>>>> err, uhdError(err)); >>>>>>>>>> return(FAIL); >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> /* Get master clock rate */ >>>>>>>>>> if ((err = uhd_usrp_get_master_clock_rate(dev[channel], 0, >>>>>>>>>> &tmp))) >>>>>>>>>> >>>>>>>>> { >>>>>>>>>> error(log, "Failed to set master clock to %.0lf Mhz >>>>>>>>>> (%d). %s.\n", 0, FL, >>>>>>>>>> LN, FN, tmp/1000000, err, uhdError(err)); >>>>>>>>>> return(FAIL); >>>>>>>>>> } >>>>>>>>>> info(stderr, "Master clock is at %.0lf Mhz\n", 0, >>>>>>>>>> tmp/1000000); >>>>>>>>>> /* Set the sample rate */ >>>>>>>>>> if (srate && !uhd_set_rx_rate_check(channel, srate)) >>>>>>>>>> return(FAIL); >>>>>>>>>> /* Set the tuner gain SBX-120 is 0-31.5 in .5 db steps */ >>>>>>>>>> >>>>>>>>> if ((err = uhd_usrp_set_rx_gain(dev[channel], gain, >>>>>>>>>> channel, ""))) >>>>>>>>>> { >>>>>>>>>> error(log, "Failed to set tuner[%d] gain to %.0lf db >>>>>>>>>> (%d). %s.\n", 0, FL, >>>>>>>>>> LN, FN, channel, gain, err, uhdError(err)); >>>>>>>>>> return(FAIL); >>>>>>>>>> } >>>>>>>>>> if (!(err = uhd_usrp_get_rx_gain(dev[channel], channel, >>>>>>>>>> "", &tmp))) >>>>>>>>>> info(log, "Tuner[%d] gain set to %.0lf (%.0lf) >>>>>>>>>> dB\n", 0, channel, tmp, gain); >>>>>>>>>> >>>>>>>>> ./* Set channel bw to conserve tuner resources. Not >>>>>>>>> needed, set by srate */ >>>>>>>>> uhd_usrp_set_rx_bandwidth(dev[channel], srate, >>>>>>>>> channel); >>>>>>>>> ./* Disable subtracting constant averaged background. >>>>>>>>> Signal looks cleaner */ >>>>>>>>> if ((err = >>>>>>>>> uhd_usrp_set_rx_dc_offset_enabled(dev[channel], false, channel))) >>>>>>>>> { >>>>>>>>> warn(log, "Failed to disable FPGA DC offset on >>>>>>>>> channel %d(%d). %s.\n", 0, >>>>>>>>> FL, LN, FN, channel, err, uhdError(err)); >>>>>>>>> } >>>>>>>>> info(stderr, "Disabled FPGA DC offset on channel >>>>>>>>> %d\n", 0, channel); >>>>>>>>> return(SUCCESS); >>>>>>>>> } >>>>>>>>> >>>>>>>>> This is the generated output: >>>>>>>>> >>>>>>>>> [INFO] [UHD] linux; GNU C++ version 13.2.0; Boost_108300; >>>>>>>>> UHD_4.6.0.0+ds1-5.1ubuntu0.24.04.1 >>>>>>>>> [INFO] [X300] X300 initialization sequence... >>>>>>>>> [INFO] [X300] Maximum frame size: 8000 bytes. >>>>>>>>> [INFO] [X300] Radio 1x clock: 200 MHz >>>>>>>>> Sat Apr 26 03:33:48 2025 [00] [+] Created USRP with args >>>>>>>>> Sat Apr 26 03:33:48 2025 [00] [+] Master clock is at 200 Mhz >>>>>>>>> Sat Apr 26 03:33:48 2025 [00] [+] Tuner[0] gain set to 30 (30) dB >>>>>>>>> Sat Apr 26 03:33:48 2025 [00] [*] scanner.l:1446:main Incorrect >>>>>>>>> maxsamples (1996). Expected 19960. >>>>>>>>> Sat Apr 26 03:33:48 2025 [00] [+] Max samples/buffer[0]: 1996 >>>>>>>>> [WARNING] [0/Radio#0] Ignoring stream command for finite >>>>>>>>> acquisition of zero samples >>>>>>>>> I hope this reads OK. Maybe next time I should attach the code:) >>>>>>>>> >>>>>>>>>> TIA >>>>>>>>>> Nikos >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> USRP-users mailing list -- usrp-users@lists.ettus.com >>>>>>>>>> To unsubscribe send an email to usrp-users-le...@lists.ettus.com >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I believe that max number of samples-per-buffer is limited by MTU >>>>>>>>> size. Which is typically around 8000 or so for "jumbo frames". >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>> >>> >
_______________________________________________ USRP-users mailing list -- usrp-users@lists.ettus.com To unsubscribe send an email to usrp-users-le...@lists.ettus.com
