This is an area of interest for me as well. I’ve been working with a radio that provides VITA-49 UDP packets that always have 1024 samples. I also have downstream blocks that do moderate sized FFTs. In the case of the FFT, I need at least N inputs. In the case of the source, it must produce exactly 1024 samples per packet. Your source seems similar, in that you produce cpi samples x 5 streams.
I would definitely have 5 outputs to your block. What do those outputs produce? There are two ways to handle it. One is to produce vectors. Whether to vector or not is the question. I think your case is on the grey line of whether you should or should not. At issue are two things: 1. Do you always produce exactly N samples – this was Jeff’s criteria. I think your interface does answer this affirmatively 2. The second is, what does your downstream expect? Vector output is a different “signature” than a stream output. While there are vector-to-stream and stream-to-vector blocks you can insert to convert, they introduce needless data copying if you can’t get the interface right. The second way is to produce streams. I would call set_output_multiple(N). This method ensures that your work() method will only be called when you can produce an even multiple of N samples. Related to that is overriding the forecast() method to tell the scheduler how many inputs you need for a requested output. I’m not sure how that would work for a source, but is appropriate for a downstream block. Since your interface to the device provides you a fixed number of samples (simultaneously, on all streams), this is what I’d probably do. Data sources have particular issues. You cannot control how frequently your block’s work() method is called. What happens if you don’t service the source fast enough and have a data overrun? Note that this case must be addressed regardless if you rely on the scheduler or spawn your own thread to service the front end. Unless you implement an unbounded queue, you have to deal with overflowing the buffer that your work() method drains. Simply dropping packets creates discontinuities in the data stream that some applications can be very sensitive to. The scheduler calls idle blocks every 250ms, unless it knows you can’t run (like setting the output multiple). You can change this timeout, but since your data arrives at 400ms intervals, the default should be fine. Obviously, your downstream block(s) need to keep up, or the buffer will fill and you will drop data. Note: it is legal and moral for your work() method to block until it can produce the requested output. So let’s say you set the output multiple to 2^20 and you are called. It’s fine to make a blocking call to get the next chunk of data and not return until it arrives. The caveat to all this, which was implicit in Jeff’s directions are that the output buffer needs to be large enough for at least one N, or your flowgraph will fail. Doing your own buffering of the get_iq_online() results trades memory for cycles (and a little complexity). My personal preference is to defer to the infrastructure rather than re-implement, so I’d be inclined combine set_min_output_buffer(N) (or possibly some multiple of N?) with set_output_multiple(N) and rely on being able to pass even multiples of get_iq_online(). Caveat: Jeff is much smarter than me about GNU Radio (you probably are too). Large buffers could be an issue. I don’t know if your 2^20 hits that threshold or not. Creating 5 outputs means you need 5 2^20 outputs rather than 10^20 as a contiguous output. Also realize that a vector of size 2^20 counts as one item. Using vectors could put pressure on the buffer allocation machinery. --- Jim Melton From: Discuss-gnuradio <discuss-gnuradio-bounces+jim.melton=sncorp....@gnu.org> On Behalf Of Carl Laufer Sent: Wednesday, August 17, 2022 07:10 To: Jeff Long <willco...@gmail.com> Cc: GNURadio Discussion List <discuss-gnuradio@gnu.org> Subject: [EXTERNAL] Re: Developing KrakenSDR Source Oh, maybe the confusion is over how many items you need to output at one time. You can hold on to your buffer - one get_iq_online() worth - until it is empty, through multiple work() calls. Copy out min(amount_left_from_fetch, output_items) and return the number of items you copied (not the max size given to work() by the scheduler). It's OK if the scheduler says to output 1024 items and you only output 4 (bad example) if that's what is most efficient, or if you have only 4 items left internally. Ah nice I think this was the missing piece of my knowledge. New stream implementation seems to be working now. Thanks! On Wed, Aug 17, 2022 at 2:30 PM Jeff Long <willco...@gmail.com<mailto:willco...@gmail.com>> wrote: Oh, maybe the confusion is over how many items you need to output at one time. You can hold on to your buffer - one get_iq_online() worth - until it is empty, through multiple work() calls. Copy out min(amount_left_from_fetch, output_items) and return the number of items you copied (not the max size given to work() by the scheduler). It's OK if the scheduler says to output 1024 items and you only output 4 (bad example) if that's what is most efficient, or if you have only 4 items left internally. On Tue, Aug 16, 2022 at 10:19 PM Jeff Long <willco...@gmail.com<mailto:willco...@gmail.com>> wrote: Output buffer size is adjustable - set_min_output_buffer(min_items) will give a buffer that is at least num_items in size, but is often larger due to alignment requirements. I wouldn't use vectors just to get around buffer sizes. Very large buffers may not work due to the way they are allocated. Give this a try first. On Tue, Aug 16, 2022 at 9:24 PM Carl Laufer <ad...@rtl-sdr.com<mailto:ad...@rtl-sdr.com>> wrote: Thanks. I think it has to be a vector output. It seems that if I'm using a stream output, and have decimation blocks downstream, output_items in the source is always smaller than cpi_size, and I can't fit the 2^20 array into output_items. I think it expects the source to adjust its output buffer size? I'd have to throw away data as there's no way to tell heimdall at runtime to reconfigure to use a smaller cpi_size. Unless, is there any way to force output_items to always be [5, cpi_size] when using a stream output in the source block? On Wed, Aug 17, 2022 at 2:13 AM Jeff Long <willco...@gmail.com<mailto:willco...@gmail.com>> wrote: Hi Carl, Use vectors only if data always needs to be grouped in exact quantities, e.g., if the GR flowgraph needs to always handle blocks of 2^20 items. In general, a 5-channel stream would be more flexible. The variation in the number of items would be due to the output buffer sometimes being empty and sometimes not. This depends on what is happening in downstream blocks, and also on random scheduling of threads. Hope that answers some of your questions. On Tue, Aug 16, 2022 at 9:49 AM Carl Laufer <ad...@rtl-sdr.com<mailto:ad...@rtl-sdr.com>> wrote: Hi All, I'm currently working on a GNU Radio source block for the KrakenSDR. So far my block mostly seems to work as expected, but I'm having some minor issues and questions. If you didn't know, the KrakenSDR is 5 RTL-SDR receivers, on the same clock with a noise source for coherence calibration of the channels. We're using it for applications like radio direction finding and passive radar, and mostly write our own code in Python. But having a GNU Radio source would be useful for others. With KrakenSDR there is a DAQ software called "heimdall" which handles all the coherent calibration automatically. In my source block, I'm able to successfully receive the data in the GNU Radio source block from heimdall via a socket connection. First so you know, the heimdall DAQ buffers an array of "cpi_size" (cpi = coherent processing interval) IQ data per channel, and outputs those arrays on the socket when it's filled. By default the cpi_size = 2^20. So in my GNU Radio source I'm receiving five, 2^20 long arrays of coherent complex IQ data every ~400ms. I believe in GNU Radio this is considered a vector? So should I make the output of the source block five port vectors, with out_sig=[(np.complex64, cpi_size)] * numChannels and set vlen to cpi_size in the yaml? Or instead should I have it as an output stream out_sig=[np.complex64] * numChannels, and be using Stream->Vector blocks when needed, with num_items set to cpi_size? I've tried both methods, and they both work. But I don't understand why when using the vector output implementation, the shape of output_items keeps flipping between (5, 2, 1048576) and (5, 1, 1048576)? Code is all at https://github.com/krakenrf/gr-krakensdr if anyone would care to take a look. Everything in Python. If anyone has any tips or comments please let me know. Thanks to anyone for your insights. Regards, Carl Laufer CONFIDENTIALITY NOTICE - SNC EMAIL: This email and any attachments are confidential, may contain proprietary, protected, or export controlled information, and are intended for the use of the intended recipients only. Any review, reliance, distribution, disclosure, or forwarding of this email and/or attachments outside of Sierra Nevada Corporation (SNC) without express written approval of the sender, except to the extent required to further properly approved SNC business purposes, is strictly prohibited. If you are not the intended recipient of this email, please notify the sender immediately, and delete all copies without reading, printing, or saving in any manner. --- Thank You.
