As an aside…. I wonder if the original USRP2 transport code is still captured in the public domain. This used a proprietary Ethernet transport for IQ before we switched to UDP in UHD transport.
Sent from my iPhone > On Jul 13, 2024, at 7:33 AM, Marcus Müller <mmuel...@gnuradio.org> wrote: > > Hi Walter, > > interesting project! > > The libpcap approach seems to be reasonable; backintheday, I used to capture > at Ethernet frame level using socket(PF_PACKET,…), but that's pretty > non-portable and comes with its own can of worms. pcap's the way to go there, > I'd say, unless you want add a protocol family to your operating system > (don't know whether that is Linux or Mac OS), which I kind of doubt. > > However: > The PUB/SUB scheme is almost certainly not what you want here – that is for > broadcasting data to multiple subscribers (or dropping them, when the > subscriber(s) aren't ready) from potentially multiple transmitters. You might > spot the problem here! If you attach a waterhose on one end, and the other > end doesn't fetch packets in intervals short enough for the receive buffer to > not overflow, these packets will just silently be dropped - business as usual > for a PUBlisher! Try the PUSH/PULL pattern: GNU Radio by principle will need > a block like the SUB block to fetch data as available, and call it back later > at some point. That will not work well in this use case. > > So, to your core question, writing a GNU Radio block for your device is > relatively easy, probably; data rates aren't *that* high, so an extra memory > copy here and there is something I'd live with for a prototype. > Methodology would be this, roughly: > 1. make an out-of-tree module. We cover this on > https://tutorials.gnuradio.org , specifically in [1]. In short, `gr_modtool > newmod yourmodname`. > 2. Make a source block (`gr_modtool add -t source -l c++ hose_source`) > 3. in the generated lib/something_impl.cc, add a `bool hose_source::start() > {}`, and also add tht `bool start() override;` method prototype to the _impl.h > 4. in the _impl.h add private fields: a set of buffers, one for each channel, > where you'll put the data "deinterleavedly" from the NIC. Make each buffer > some (say, 2²⁰) GNSS samples long. Also add two unsigned integer counters: > one read and one write index. And because we're lazy and don't care *that* > much about performance yet, two mutexes (one for securing access to the read > index, and one for the write index). > 5. in the constructor, you allocate these buffers, set the read index to the > length of the buffers (-1) and the write index to 0 > 6. in the start() method, you spawn a thread that, in a loop > 1. checks how much space there is between read and write index (get read > mutex, fetch read index value, release mutex, calculate difference) > 2. uses pcap to fetch packets, (but only as much as the space calculated > above allows for!), deinterleaves data onto the buffers, finally > 4. updates write index (get write mutex, update write index, release write > mutex) > 7. the block's work() method is called by GNU Radio regularly and > 1. checks how much data is between write and read indices (get write mutex, > read write index, release mutex, calculate difference) > 2. checks whether that's more or less than the space for output items > available in this current call to work(), takes the minimum of both > 3. gets that amount of items from each buffer and writes them to the output > buffer, as passed as argument to the work() method (you could do type > conversions here!) > 4. updates the read index accordingly (get read mutex, update index, release > mutex) > 5. returns the number of written items > > note that the index updating and distance calculation need to take the > "wraparound" at the end of the buffer into account. > > Also note: very similarly, you could write a **SoapySDR** driver instead of a > GNU Radio block. You could then use the Generic SoapySDR Source block to get > data from that driver, and other, non-GNU Radio programs, could be using the > driver just as well, without knowing about the hardware. I don't think the > basic principle would be much different: you need an IO thread that keeps the > NIC busy, and because readers might be slow, an internal buffer, which you > ideally use constructively (instead of just incurring a memory bandwidth > overhead), to deinterleave channels on ingress, and to convert data types on > egress, if you will. > > Note that one *could* potentially, as mentioned above write a zero-copy-ish > driver for GNU Radio 3.10+ using our custom buffer framework and something > like AF_XDP, dpdk, or io_uring, but I think that would very much a) leave the > scope of what anyone be able to assist you with – to the best of my > knowledge, you'd be the first to do that with a network device – and b) we're > "only" talking gigabit ethernet here, and you got a fast machine. As you said > in your email, in principle, things are plenty fast enough, so let's not > overcomplicate. > > [1] > https://wiki.gnuradio.org/index.php?title=Creating_C%2B%2B_OOT_with_gr-modtool > >> On 12.07.24 22:42, Walter Szeliga wrote: >> Hi all, >> I have a GNSS Firehose >> (https://transitiva.com/product/gnss-firehose-receiver-tri-band-quad-constellation/ >> >> <https://transitiva.com/product/gnss-firehose-receiver-tri-band-quad-constellation/>) >> and have been trying to get it working, in a streaming capacity, with >> Gnuradio. The Firehose sends packets over ethernet using the experimental >> ethertype 0x88b5. I've tried a few things to get data from the Firehose into >> Gnuradio, some have worked, others have not. Things that work: >> * Use tcpdump and filter on 0x88b5 and save to a file. Open and repackage >> each packet in the pcap dump as interleaved bytes of I&Q and save to a file. >> Read into Gnuradio. >> * Write a custom program using libpcap to filter on 0x88b5 on a selected >> interface, repackage the packets and write directly to a file. Read into >> Gnuradio. >> * Write a custom program using libpcap to filter on 0x88b5 on a selected >> interface, repackage the packets and PUB them using ZMQ. SUB to this PUB >> using a simple Python script and dump the message contents to a file. Read >> into Gnuradio. Both tcp and ipc PUB/SUB work equally well. >> Some things that do not work: >> * SUB to the ZMQ PUB/SUB using the Gnuradio ZMQ SUB Source block. >> * Write a custom program using libpcap to filter on 0x88b5 on a selected >> interface, repackage the packets and send them using UDP. >> The UDP approach doesn't work because too many packets get dropped and I >> have been unable to set sysctl values appropriately (on an M1 Mac) to avoid >> dropping too many packets. >> I'm not sure why the ZMQ approach does not work with Gnuradio. I've tried >> many simple flowgraphs to convert from vector to stream, ibyte to complex, >> you name it, and then dumping the data back out to a file using a File Sink >> (just to use existing software to check for data sanity). Data gets into >> Gnuradio, but it clearly loses something because constellation diagrams of >> the output data become blobs centered on the origin rather than pairs of >> point clouds offset from the origin as one would expect from the BPSK nature >> of the GNSS signals captured by the Firehose. >> I've come to the realization that it's probably best to write some sort of >> driver to get data straight from the Firehose into Gnuradio, but I have no >> idea where to start with this. Any ideas about how to fix my ZMQ approach or >> start writing a custom driver would be appreciated. There's a lot in here, >> so let me know if you would like code or flowgraph examples. >> Cheers, >> Walter >
