Hi Paul, Thanks for your post. I am new to USRP devices and UHD and GNURadio but I have recently obtained an X310 with dual TwinRX daughterboards to do pulsar radio astronomy and I have been exploring exactly the issue you raise in order to help me with my requirements to record data for hours at a time.
Thus far, my results are similar to your results it seems as I am able to write dual-channel 32-bit complex samples from the X310 to disk at 25Msps without loss of any samples, or at 50Msps using a single channel for hours at a time. I’m using a single 10G ethernet connection of a dual-channel 10G interface to the PC. The PC is an AMD Threadripper 2950x CPU @3.4GHz with an Intel 2TB SSD and an 8TB HDD for storage. To achieve the above throughput I am writing directly to the SSD only as the HDD can’t keep up with these data rates. At the moment I am using a GNURadio-Companion flowchart program to control the X310 but I will shortly be exploring use of my own C++ program utilizing only UHD routines to perform the transfer task if I can arrange that. The effort to use my own program is driven by my desire to explore using 16-bit complex or even 8-bit sample sizes from the X310 to allow me to record 100Msps from dual channels but I don’t think GRC allows those choices of sample format into the File Sink block, unless I’m misunderstanding the capabilities of GRC, so I’ll attempt to write my own in C++ using UHD to do this instead. Being new to much of this I don’t know if I can be of any help to you but I thought I’d let you know that I for one am very interested in hearing of your progress as your and my goals appear to be similar with respect to achieving dual-channel, wide-bandwidth output from the X310 to disk. Regards, Joe > On Mar 9, 2019, at 4:32 AM, Paul Boven via USRP-users > <usrp-users@lists.ettus.com> wrote: > > Hi, > > I'm trying to record the full X310 bandwidth, for a few hours, without any > missed samples. Which of course is a bit of a challenge - does anyone here > already achieve this? > > We're using a TwinRX, so initially I wanted to record 2x 100MS/s (from both > channels), which amounts to 800MB/s, 6.4Gb/s. At first I tried uhd_rx_cfile, > but have been unable to get it to a good state without showing an 'O' every > few seconds at these speeds. > > As a recorder I have a SuperMicro 847 chassis with 36 disks (Seagate Ironwolf > 8TB T8000VN0022, 7200rpm). In this particular server, the disks are connected > through an 'expander' backplane, from a single HBA (LSI 3008). CPU is dual > Xeon 4110, 2.1 GHz, 64 GB of ram. > > At first I tried a 6 disk pool (raidz1), and eventually ended up creating a > huge 36 disk ZFS stripe, which in theory should have no trouble with the > throughput, but certainly kept dropping packets. > > Note that recording to /dev/shm/file works perfectly without dropping > packets, until the point that the memory is full. > > Given that ZFS has quite a bit of (good) overhead to safeguard your data, I > then switched to creating a mdadm raid-0 with 18 of the disks (Why not 36? I > was really running out of time!) > > At that point I also found 'specrec' from gr-analyze, which seems more > suitable. But, even after enlarging its circular buffer to the largest > supported values, it would only average a write speed of about 300MB/s. > > In the end I had to settle for recording at only 50MS/s (200MB/s) from only a > single channel, a far cry from the 2x 6.4Gb/s I'm ultimately looking to > record. Although I did get more than an hour of perfect data out of it, over > time the circular buffer did get fuller in bursts, and within 2 hours it > exited due to exhausting the buffers. Restarting the application made it work > like fresh again, with the same gradual decline > in performance. > > Specrec, even when tweaking its settings, doesn't really take advantage of > the large amount of memory in the server. As a next step, I'm thinking of > adapting specrec to use much larger buffers, so that writes are at least in > the range of MB to tens of MB. From earlier experiences, it is also important > to flush your data to disk often, so the interruptions due to this are more > frequent, but short enough to not cause receive buffers to overflow. > > In terms of network tuning, all recording was done with MTU 9000, with wmem > and rmem at the recommended values. All recordings were done as interleaved > shorts. > > Does anyone have hints or experiences to share? > > Regards, Paul Boven. > > _______________________________________________ > USRP-users mailing list > USRP-users@lists.ettus.com > http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com > _______________________________________________ USRP-users mailing list USRP-users@lists.ettus.com http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
