On Jul 13, 2024, at 7:33 AM, Marcus Müller <mmuel...@gnuradio.org> wrote:
Hi Walter,
interesting project!
The libpcap approach seems to be reasonable; backintheday, I used to capture at
Ethernet frame level using socket(PF_PACKET,…), but that's pretty non-portable
and comes with its own can of worms. pcap's the way to go there, I'd say,
unless you want add a protocol family to your operating system (don't know
whether that is Linux or Mac OS), which I kind of doubt.
However:
The PUB/SUB scheme is almost certainly not what you want here – that is for
broadcasting data to multiple subscribers (or dropping them, when the
subscriber(s) aren't ready) from potentially multiple transmitters. You might
spot the problem here! If you attach a waterhose on one end, and the other end
doesn't fetch packets in intervals short enough for the receive buffer to not
overflow, these packets will just silently be dropped - business as usual for a
PUBlisher! Try the PUSH/PULL pattern: GNU Radio by principle will need a block
like the SUB block to fetch data as available, and call it back later at some
point. That will not work well in this use case.
So, to your core question, writing a GNU Radio block for your device is
relatively easy, probably; data rates aren't *that* high, so an extra memory
copy here and there is something I'd live with for a prototype.
Methodology would be this, roughly:
1. make an out-of-tree module. We cover this on https://tutorials.gnuradio.org
, specifically in [1]. In short, `gr_modtool newmod yourmodname`.
2. Make a source block (`gr_modtool add -t source -l c++ hose_source`)
3. in the generated lib/something_impl.cc, add a `bool hose_source::start()
{}`, and also add tht `bool start() override;` method prototype to the _impl.h
4. in the _impl.h add private fields: a set of buffers, one for each channel, where
you'll put the data "deinterleavedly" from the NIC. Make each buffer some (say,
2²⁰) GNSS samples long. Also add two unsigned integer counters: one read and one write
index. And because we're lazy and don't care *that* much about performance yet, two
mutexes (one for securing access to the read index, and one for the write index).
5. in the constructor, you allocate these buffers, set the read index to the
length of the buffers (-1) and the write index to 0
6. in the start() method, you spawn a thread that, in a loop
1. checks how much space there is between read and write index (get read
mutex, fetch read index value, release mutex, calculate difference)
2. uses pcap to fetch packets, (but only as much as the space calculated
above allows for!), deinterleaves data onto the buffers, finally
4. updates write index (get write mutex, update write index, release write
mutex)
7. the block's work() method is called by GNU Radio regularly and
1. checks how much data is between write and read indices (get write mutex,
read write index, release mutex, calculate difference)
2. checks whether that's more or less than the space for output items
available in this current call to work(), takes the minimum of both
3. gets that amount of items from each buffer and writes them to the output
buffer, as passed as argument to the work() method (you could do type
conversions here!)
4. updates the read index accordingly (get read mutex, update index, release
mutex)
5. returns the number of written items
note that the index updating and distance calculation need to take the
"wraparound" at the end of the buffer into account.
Also note: very similarly, you could write a **SoapySDR** driver instead of a
GNU Radio block. You could then use the Generic SoapySDR Source block to get
data from that driver, and other, non-GNU Radio programs, could be using the
driver just as well, without knowing about the hardware. I don't think the
basic principle would be much different: you need an IO thread that keeps the
NIC busy, and because readers might be slow, an internal buffer, which you
ideally use constructively (instead of just incurring a memory bandwidth
overhead), to deinterleave channels on ingress, and to convert data types on
egress, if you will.
Note that one *could* potentially, as mentioned above write a zero-copy-ish driver for
GNU Radio 3.10+ using our custom buffer framework and something like AF_XDP, dpdk, or
io_uring, but I think that would very much a) leave the scope of what anyone be able to
assist you with – to the best of my knowledge, you'd be the first to do that with a
network device – and b) we're "only" talking gigabit ethernet here, and you got
a fast machine. As you said in your email, in principle, things are plenty fast enough,
so let's not overcomplicate.
[1]
https://wiki.gnuradio.org/index.php?title=Creating_C%2B%2B_OOT_with_gr-modtool
On 12.07.24 22:42, Walter Szeliga wrote:
Hi all,
I have a GNSS Firehose
(https://transitiva.com/product/gnss-firehose-receiver-tri-band-quad-constellation/
<https://transitiva.com/product/gnss-firehose-receiver-tri-band-quad-constellation/>)
and have been trying to get it working, in a streaming capacity, with Gnuradio. The
Firehose sends packets over ethernet using the experimental ethertype 0x88b5. I've
tried a few things to get data from the Firehose into Gnuradio, some have worked,
others have not. Things that work:
* Use tcpdump and filter on 0x88b5 and save to a file. Open and repackage each
packet in the pcap dump as interleaved bytes of I&Q and save to a file. Read
into Gnuradio.
* Write a custom program using libpcap to filter on 0x88b5 on a selected
interface, repackage the packets and write directly to a file. Read into
Gnuradio.
* Write a custom program using libpcap to filter on 0x88b5 on a selected
interface, repackage the packets and PUB them using ZMQ. SUB to this PUB using
a simple Python script and dump the message contents to a file. Read into
Gnuradio. Both tcp and ipc PUB/SUB work equally well.
Some things that do not work:
* SUB to the ZMQ PUB/SUB using the Gnuradio ZMQ SUB Source block.
* Write a custom program using libpcap to filter on 0x88b5 on a selected
interface, repackage the packets and send them using UDP.
The UDP approach doesn't work because too many packets get dropped and I have
been unable to set sysctl values appropriately (on an M1 Mac) to avoid dropping
too many packets.
I'm not sure why the ZMQ approach does not work with Gnuradio. I've tried many
simple flowgraphs to convert from vector to stream, ibyte to complex, you name
it, and then dumping the data back out to a file using a File Sink (just to use
existing software to check for data sanity). Data gets into Gnuradio, but it
clearly loses something because constellation diagrams of the output data
become blobs centered on the origin rather than pairs of point clouds offset
from the origin as one would expect from the BPSK nature of the GNSS signals
captured by the Firehose.
I've come to the realization that it's probably best to write some sort of
driver to get data straight from the Firehose into Gnuradio, but I have no idea
where to start with this. Any ideas about how to fix my ZMQ approach or start
writing a custom driver would be appreciated. There's a lot in here, so let me
know if you would like code or flowgraph examples.
Cheers,
Walter
