Hi Jim, >From a quick review of this chain, it appears your concern is for the transmit direction (hence, Underruns). Although I have never tried to run for extended periods such as you are requiring, I am reasonably confident that the issue is on the host side and not the Radio side. Some remarks:
- Perhaps you would get more reliable streaming using DPDK. When I last tried this a couple of years ago, the performance was great (at much higher rates than you, but for shorter periods of time). But, the side effects were substantial (PC mis-behaving to the point of becoming unusable) and so I abandoned it. The new DPDK is more mature (as is the OS) from when I tried it so it may be much better now with regard to side effects. - You could add buffering on the N310 (by building a custom FPGA image) that could robustly handle short host "hiccups" in streaming. Originally, the DMA-FIFO RFNoC block was put in the Tx path of devices such as the X310 for this express purpose. But, the DMA-FIFO block cannot handle 4 streams at full rate on an N310 so it is not part of the stock FPGA. The external RAM is used instead for the "Replay" block. But at your data rates, the external RAM can handle four streams so you could build an FPGA image that replaced the Replay block with the DMA-FIFO such that you would have very large buffers on the FPGA to handle host streaming hiccups. Note that you could also build a new image with larger FPGA fabric buffers, but they wouldn't approach the size of the external RAM. If building your own FPGA image seems daunting (for some this is a show stopper), I just want to mention that what I am suggesting would not require FPGA design talent - the necessary blocks already exist - it would just require purchasing Xilinx Vivado and getting past the initial learning curve of building an image. - How are you generating your Tx samples? Custom app? Reading from a very large data file? I ask this because I have had the most streaming success when transmitting from a file (or receive to a file) if the file is in a "RAM disk". We generally order Linux PCs with as much RAM as we can afford in order to configure such RAM as a RAM disk for the purpose of fast streaming to/from files. - Finally, if your data is not dynamic (a repeated fixed length sequence such as in a repeated radar transmission), I would highly recommend using the Replay block to send the data rather than streaming from the host. You will likely never see an underrun in this case. But, I realize that if the data is dynamic, this is not possible. Rob On Thu, Apr 13, 2023 at 5:03 PM Jim Schatzman < james.schatz...@futurelabusa.com> wrote: > Even with all the recommended settings, and a very fast computer that is > doing nothing except sending the data, it is maybe 50/50 that a 2 hour > simulation can be conducted without an underrun. The longest run I have > been able to do without an underrun is about 2.5 hours. > > The sample rate is 12.5 Msamp/sec at 16 bit I + 16 bit Q or 400 Mbit/sec. > > For our application, that is unacceptable. I need to be able to run for > days without data loss. > > It is a mystery to me why a 10 Gbit connection cannot support 400 Mbit/sec > UDP reliably. > > Any ideas about how we can completely eliminate underruns? > > At the moment, I am uncertain whether the problem is occurring on the host > or on the radio. I suspect the radio, but I will do some testing of the > host to see what UDP data rate it can support without loss. > > Thanks! > > > > > > At 10:53 PM 4/10/2023, Marcus D. Leech wrote: > >On 10/04/2023 21:12, Jim Schatzman wrote: > >>The following steps had no impact: > >> > >>a) Don't use a switch but do a point-to-point connection between the > comptuer's NIC and the N310. > >>b) Adjust network buffers and ring buffer per recommendations (actually, > the network buffers setting was always being done). > >> > >>Increasing the MTU to 9000 had a significant impact. An occasionaly > underrun is still experienced, but an hour or two of continuous > transmission is possible. > >> > >>I am wondering if this is not an issue on the computer side but on the > radio side. Is the N310 incapable of supporting 1x 10 Gbps streams with a > MTU of 1500? Perhaps. > >> > >>I will do some computer-to-computer testing to see if the problem can be > reproduced there. > >> > >>Jim > >Even non-SDR applications tend to use jumbo-frames for continuous traffic > at 10Gbit. I'm sorry that I didn't clue in to that > > in the first round. > > > > > >> > >> > >> > >> > >>At 08:39 PM 4/7/2023, Marcus D. Leech wrote: > >>>On 07/04/2023 22:13, Jim Schatzman wrote: > >>>>We have been unable to estable 100% reliable connections to an N310 > USRP radio through its 10 Gbit ethernet from Linux. What happens is that it > works fine for a period of time - 30 to 60 minutes, typically. Then we see > a couple of U's in the output. Unfortunately, that is fatal for our > application. > >>>> > >>>>Using the unmodified tx_samples_from_file or one modified to use > separate threads to read data from the file and to sent it over the socket > to the radio, the symptoms are the same. > >>>> > >>>>All the evidence is that the application is sending data continuously > without any delays. Also, the "network" has no devices on it except for the > host computer, a high performance 10G switch, and the N310 radio. > >>>> > >>>>We are wondering if this could be a Linux "feature". We would like to > try increasing the socket priority with SO_PRIORITY. However, we are not > finding any hooks in the UHD software for this. > >>>> > >>>>Suggestions? > >>>> > >>>>Thanks! > >>>>Jim > >>>>_______________________________________________ > >>>Have you increased the ring buffers on your card? > >>> > >>> > https://kb.ettus.com/USRP_Host_Performance_Tuning_Tips_and_Tricks#Increasing_Ring_Buffers > >>> > >>>Also, adjust the network buffers: > >>> > >>> > https://kb.ettus.com/USRP_Host_Performance_Tuning_Tips_and_Tricks#Adjust_Network_Buffers > >>> > >>>Have you tried a direct connection--without the switch?   Just > to eliminate it. Before you say "oh, it's not the switch", > >>> many "high performance" 10G switches have a less-than-ideal > switching fabric. > >>> > >>>_______________________________________________ USRP-users mailing list > -- usrp-users@lists.ettus.com To unsubscribe send an email to > usrp-users-le...@lists.ettus.com </x-flowed> > >>_______________________________________________ > >>USRP-users mailing list -- usrp-users@lists.ettus.com > >>To unsubscribe send an email to usrp-users-le...@lists.ettus.com > >_______________________________________________ USRP-users mailing list > -- usrp-users@lists.ettus.com To unsubscribe send an email to > usrp-users-le...@lists.ettus.com </x-flowed> > _______________________________________________ > USRP-users mailing list -- usrp-users@lists.ettus.com > To unsubscribe send an email to usrp-users-le...@lists.ettus.com >
_______________________________________________ USRP-users mailing list -- usrp-users@lists.ettus.com To unsubscribe send an email to usrp-users-le...@lists.ettus.com
