On Jul 13, 2024, at 7:33 AM, Marcus Müller <mmuel...@gnuradio.org> wrote:
Hi Walter,
interesting project!
The libpcap approach seems to be reasonable; backintheday, I used to capture at
Ethernet frame level using socket(PF_PACKET,…), but that's pretty non-portable and
comes with its own can of worms. pcap's the way to go there, I'd say, unless you want
add a protocol family to your operating system (don't know whether that is Linux or Mac
OS), which I kind of doubt.
However:
The PUB/SUB scheme is almost certainly not what you want here – that is for
broadcasting data to multiple subscribers (or dropping them, when the subscriber(s)
aren't ready) from potentially multiple transmitters. You might spot the problem here!
If you attach a waterhose on one end, and the other end doesn't fetch packets in
intervals short enough for the receive buffer to not overflow, these packets will just
silently be dropped - business as usual for a PUBlisher! Try the PUSH/PULL pattern: GNU
Radio by principle will need a block like the SUB block to fetch data as available, and
call it back later at some point. That will not work well in this use case.
So, to your core question, writing a GNU Radio block for your device is relatively
easy, probably; data rates aren't *that* high, so an extra memory copy here and there
is something I'd live with for a prototype.
Methodology would be this, roughly:
1. make an out-of-tree module. We cover this on https://tutorials.gnuradio.org ,
specifically in [1]. In short, `gr_modtool newmod yourmodname`.
2. Make a source block (`gr_modtool add -t source -l c++ hose_source`)
3. in the generated lib/something_impl.cc, add a `bool hose_source::start() {}`, and
also add tht `bool start() override;` method prototype to the _impl.h
4. in the _impl.h add private fields: a set of buffers, one for each channel, where
you'll put the data "deinterleavedly" from the NIC. Make each buffer some (say, 2²⁰)
GNSS samples long. Also add two unsigned integer counters: one read and one write
index. And because we're lazy and don't care *that* much about performance yet, two
mutexes (one for securing access to the read index, and one for the write index).
5. in the constructor, you allocate these buffers, set the read index to the length of
the buffers (-1) and the write index to 0
6. in the start() method, you spawn a thread that, in a loop
1. checks how much space there is between read and write index (get read mutex, fetch
read index value, release mutex, calculate difference)
2. uses pcap to fetch packets, (but only as much as the space calculated above allows
for!), deinterleaves data onto the buffers, finally
4. updates write index (get write mutex, update write index, release write
mutex)
7. the block's work() method is called by GNU Radio regularly and
1. checks how much data is between write and read indices (get write mutex, read
write index, release mutex, calculate difference)
2. checks whether that's more or less than the space for output items available in
this current call to work(), takes the minimum of both
3. gets that amount of items from each buffer and writes them to the output buffer,
as passed as argument to the work() method (you could do type conversions here!)
4. updates the read index accordingly (get read mutex, update index, release
mutex)
5. returns the number of written items
note that the index updating and distance calculation need to take the "wraparound" at
the end of the buffer into account.
Also note: very similarly, you could write a **SoapySDR** driver instead of a GNU Radio
block. You could then use the Generic SoapySDR Source block to get data from that
driver, and other, non-GNU Radio programs, could be using the driver just as well,
without knowing about the hardware. I don't think the basic principle would be much
different: you need an IO thread that keeps the NIC busy, and because readers might be
slow, an internal buffer, which you ideally use constructively (instead of just
incurring a memory bandwidth overhead), to deinterleave channels on ingress, and to
convert data types on egress, if you will.
Note that one *could* potentially, as mentioned above write a zero-copy-ish driver for
GNU Radio 3.10+ using our custom buffer framework and something like AF_XDP, dpdk, or
io_uring, but I think that would very much a) leave the scope of what anyone be able to
assist you with – to the best of my knowledge, you'd be the first to do that with a
network device – and b) we're "only" talking gigabit ethernet here, and you got a fast
machine. As you said in your email, in principle, things are plenty fast enough, so
let's not overcomplicate.
[1]
https://wiki.gnuradio.org/index.php?title=Creating_C%2B%2B_OOT_with_gr-modtool
On 12.07.24 22:42, Walter Szeliga wrote:
Hi all,
I have a GNSS Firehose
(https://transitiva.com/product/gnss-firehose-receiver-tri-band-quad-constellation/
<https://transitiva.com/product/gnss-firehose-receiver-tri-band-quad-constellation/>)
and have been trying to get it working, in a streaming capacity, with Gnuradio. The
Firehose sends packets over ethernet using the experimental ethertype 0x88b5. I've
tried a few things to get data from the Firehose into Gnuradio, some have worked,
others have not. Things that work:
* Use tcpdump and filter on 0x88b5 and save to a file. Open and repackage each packet
in the pcap dump as interleaved bytes of I&Q and save to a file. Read into Gnuradio.
* Write a custom program using libpcap to filter on 0x88b5 on a selected interface,
repackage the packets and write directly to a file. Read into Gnuradio.
* Write a custom program using libpcap to filter on 0x88b5 on a selected interface,
repackage the packets and PUB them using ZMQ. SUB to this PUB using a simple Python
script and dump the message contents to a file. Read into Gnuradio. Both tcp and ipc
PUB/SUB work equally well.
Some things that do not work:
* SUB to the ZMQ PUB/SUB using the Gnuradio ZMQ SUB Source block.
* Write a custom program using libpcap to filter on 0x88b5 on a selected interface,
repackage the packets and send them using UDP.
The UDP approach doesn't work because too many packets get dropped and I have been
unable to set sysctl values appropriately (on an M1 Mac) to avoid dropping too many
packets.
I'm not sure why the ZMQ approach does not work with Gnuradio. I've tried many simple
flowgraphs to convert from vector to stream, ibyte to complex, you name it, and then
dumping the data back out to a file using a File Sink (just to use existing software
to check for data sanity). Data gets into Gnuradio, but it clearly loses something
because constellation diagrams of the output data become blobs centered on the origin
rather than pairs of point clouds offset from the origin as one would expect from the
BPSK nature of the GNSS signals captured by the Firehose.
I've come to the realization that it's probably best to write some sort of driver to
get data straight from the Firehose into Gnuradio, but I have no idea where to start
with this. Any ideas about how to fix my ZMQ approach or start writing a custom driver
would be appreciated. There's a lot in here, so let me know if you would like code or
flowgraph examples.
Cheers,
Walter
