On 8/16/16 11:28 PM, Inspire Me wrote:
Hi Ed
Much appreciated. You mentioned latency due to queued flags. This has
potential to cause us issues. I was wondering if it is possible to build
a block as a sort of switch that takes input from the standard HDLC
Framer but executues continuously, ie once every symbol period T=
n*(1/(9600*4), it would check to see if any items are in the input
buffer and if not push nFlags out (n being small), boost sleep for on
n*symbol period and repeat. I know this would be highly inefficient but
given it will be a very basic block it would only execute for a fraction
of the symbol time. The other option was to have a seperate thread in
the block for sending the flags which runs independent of the Work
function and use a semaphore to enable / disable the sending of flags..
Your thoughts and insight would be valuable.
Again, I repeat that GnuRadio blocks are timing agnostic. You cannot
"pace" the bitstream in real-time by the use of "sleeps".
You need to write a block that has NO bitstream inputs, and one
bitstream output. This is known as a "source" block. In addition
it should have a message port for receiving HDLC frames. As in
the example I showed you, this new block should check the message
port for an HDLC frame, bitstuff it into the output bitstream, and
repeat this until either the message port is empty or the output
buffer is full. If the message port is empty on the initial entry
to work, output a single FLAG worth of bits and exit, returning a
value of 8.
Q.I thought I read that the schedular only schedules work function when
there is input data to process. In other words if no input is available
then the work function would not be scheduled. Is this correct ? If not
where can I find more information on the scheduler. I have read material
available online including (
https://static.squarespace.com/static/543ae9afe4b0c3b808d72acd/543aee1fe4b09162d0863397/543aee20e4b09162d0863578/1380223973117/gr_scheduler_overview.pdf
)
The scheduler will schedule a block to run if there is input available
AND the output buffer is available. That is, all of the immediately
downstream blocks have indicated that they have consumed the data.
This is called backpressure. In the case of a source block, there
are no inputs, and the block is run whenever the output buffer space is
available.
Once again thank you for spending time helping.
No problem. I believe you said you're doing a CubeSat application?
On small satellites, often the best approach to this problem is to
use an HDLC framer chip, which does all this for you and produces
a simple bitstream with flags and frames all in their proper places.
This bitstream is then straightforward to handle and send to a modulator.
One last note. Keep the discussion on-list so others may benefit
(do a "Reply All" instead of a "Reply", otherwise it just comes to me).
@(^.^)@ Ed
Kind Regards
Frank
On 17 August 2016 at 11:59, Ed Criscuolo <e...@chessie.com
<mailto:e...@chessie.com>> wrote:
In a GnuRadio block that is derived from a "sync" block, the number of
input and output items are the same, so the work function is supplied
with a single value called "nitems". The normal control flow of a
typical work function uses a loop that iterates over nitems, taking
one item from the input(s) buffer(s) and producing one item into the
output buffer. This process repeats until all input items are consumed
and all output buffer item spaces are full. The work function then
returns a value of "nitems" to indicate to the scheduler how many items
were consumed and produced.
But it is perfectly legal for the loop in the work function to
terminate early, in which case it should return the number of items
_actually_ processed. The scheduler will take care of keeping track
of the partial buffers and no data will be lost. But performance
will be reduced.
GnuRadio blocks are timing agnostic. They run as fast as they can,
and process an entire buffer's worth of items (typically 64K items)
per call to the work function. This is done to improve throughput.
Timing only happens when your bitstream meets up with a hardware
device, like a USRP. But as long as the _average_ throughput is
fast enough to keep up with the hardware, everything is fine.
The amount of CPU processing "headroom" determines how far you can
degrade a block's average throughput without creating underruns.
This level of understanding is fundamental to writing any GnuRadio
blocks, so you really should take the time to play with it and get
to know it.
Here is a snippet of VERY OLD code for example only. It that always
runs "nitems" times. When idle (ie - no hdlc packets available), it
produces 64k bits (8k of flags) on every invocation! This results in
a large latency when a packet finally does come along.
It should be modified to break the loop on the second fifo empty
test and return "i" instead of "nitems"
int
sc_hdlc_router_source_b::work (int noutput_items,
gr_vector_const_void_star
&input_items,
gr_vector_void_star
&output_items)
{
unsigned char * outbuf = (unsigned char *) output_items[0];
int i;
// Loop until the requested number of output stream bytes have
been generated
for(i=0; i<noutput_items; i++)
{
if (d_fifo.empty())
{
// Process a single incoming packet on the tun/tap pseudo
network device
// by bitstuffing it, encapsulating it in an MPoFR frame,
and pushing
// it into the bit_buf fifo.
encapsulate_incoming_packet();
}
// If bit_buf fifo is still empty, push a FLAG into it
if(d_fifo.empty())
{
push_flag();
}
// Output a bit from the bitbuf in the low-order bit of the
output byte
outbuf[i] = d_fifo.pop();
}
return noutput_items;
}
@(^.^)@ Ed
On 8/16/16 8:12 PM, Inspire Me wrote:
Hi Ed
Thank you for the response.
I like the idea of the second option. Could please expand the
explaination. I am new to gnuradio and don't yet fully grasp how the
scheduler works in detail. How do you terminate the work
function early
and ensure the single 0x7E gets streamed immediately to the
modulator so
that we don't get under runs. How quickly will the work function
be re
executed by the scheduler ? As fas as CPU is concerned, I have a
machine
with 16 cores ( 32 threads). I am not concened if one thread is
consumed
by this. Could you send me a snippet of the work function you
implemented.
Your assitance is much appreciated.
Kind Regards
Frank
On 17 August 2016 at 02:00, Ed Criscuolo <e...@chessie.com
<mailto:e...@chessie.com>
<mailto:e...@chessie.com <mailto:e...@chessie.com>>> wrote:
Hi Frank,
I'm the author of the GMSK Spacecraft Groundstation project you
referenced. Been on vacation for the last 2 weeks without
internet,
so I'm just seeing this.
All in all, Marcus is giving you good advice. At the time
I wrote
it, there were no tags or messages, so everything related to
PDU-like behavior had to be implemented in a single C++
block. Now,
packets can come in as PDUs, and get converted into a
bitstream in a
single, much simpler block.
As for the issue of putting HDLC flag octets out
back-to-back as the
"idle" pattern between frames, I had tried two different
solutions,
one of which Marcus touched upon.
First approach was to simply reduce the buffer size of all
blocks by
recompiling GnuRadio with the appropriate global constant
reset to a
smaller value. This has significant impact on overall
performance,
but because our data rate was modest (only a few hundred
Kbits/sec)
and we had some wicked fast hardware (for the times!) we
were able
to get this to work, reducing the added latency of outbound
packets
from over ten seconds to only a few seconds. This was OK
for our
particular requirements, but not for a general purpose
system. It
was an easy but VERY brute-force solution.
Later, I modified the block to to implement a solution
similar to
Marcus' "Option A". Every time the block's work function
was entered,
it checked for an available packet (I was using sockets,
but modern
GnuRadio would use PDUs and message queues). If a packet
was available,
it would be bitstuffed and the bitstream written to the output
buffer. This would continue until either no more packets were
available, or the
output buffer reached capacity. If there were NO packets
available
when the work function was called, it would put a SINGLE
0x7E (that
is, a bitstream of eight bits) on the output buffer and
terminate
the work function early, returning only the numbers of bits
actually
produced.
This is perfectly allowable, but it will result in the
block consuming
a large amount of CPU when generating idle flags. At least
this
solution only effects a single block instead of all the blocks.
If the resulting CPU usage is too high, it could be
reduced, at the
cost of adding latency, by outputting 2 or more flags at a
time.
Hope this helps.
@(^.^)@ Ed
On 8/15/16 7:00 PM, Inspire Me wrote:
Hi Nick / Marcus
Great discussion, obviously this issue has been solved
before, as
mentioned in the 2009 discussion the
GMSKSpacecraftGroundstation
project
(
https://moo.cmcl.cs.cmu.edu/trac/cgran/wiki/GMSKSpacecraftGroundstation
<https://moo.cmcl.cs.cmu.edu/trac/cgran/wiki/GMSKSpacecraftGroundstation>
<https://moo.cmcl.cs.cmu.edu/trac/cgran/wiki/GMSKSpacecraftGroundstation
<https://moo.cmcl.cs.cmu.edu/trac/cgran/wiki/GMSKSpacecraftGroundstation>>
<https://moo.cmcl.cs.cmu.edu/trac/cgran/wiki/GMSKSpacecraftGroundstation
<https://moo.cmcl.cs.cmu.edu/trac/cgran/wiki/GMSKSpacecraftGroundstation>
<https://moo.cmcl.cs.cmu.edu/trac/cgran/wiki/GMSKSpacecraftGroundstation
<https://moo.cmcl.cs.cmu.edu/trac/cgran/wiki/GMSKSpacecraftGroundstation>>>
)
seems to have solved it. I acknowledge that gnuradio
has moved
on, but I
would like to see how it was solved and possibly adapt
it to the
current
gnuradio. *Is there anyway to get access to the code ?*
I am
stuck at
the moment.
The only idea I have is to impelement a source block
with very small
buffers to feed into the Tx Chain and some sort of
switch to toggle
between data stream and flag stream based on a message
/ tag
from up the
chain.
Nick can you elaborate on the modification of the
Work() function. I
thought this was scheduled only when sufficient data is
in the Input
buffer ? Is there a way to have this execute more often
or when
no data
exists in the input buffer ? Is there a way to implement a
switch block
that monitors input streams and efectively switches the
output
stream
such that it is not blocked waiting for input from one
of the
streams ?
Once again thank you for the help
Kind Regards
Frank
On 16 August 2016 at 03:14, Marcus Müller
<marcus.muel...@ettus.com
<mailto:marcus.muel...@ettus.com>
<mailto:marcus.muel...@ettus.com <mailto:marcus.muel...@ettus.com>>
<mailto:marcus.muel...@ettus.com
<mailto:marcus.muel...@ettus.com>
<mailto:marcus.muel...@ettus.com
<mailto:marcus.muel...@ettus.com>>>> wrote:
Hi Nick,
On 15.08.2016 19:19, Nick Foster wrote:
This is a pretty familiar problem. A lot of satcom
systems require
continuous transmission when in an idle state
so the
receiver can
use slow (i.e., sensitive) frequency and timing
estimator loops.
I'm sorry to say there's no great answer.
But you have some options:
a) modify the HDLC block so that when work()
is called,
if nothing
is in the msg queue just output some fixed
number of
0x7Es and sit
on its thumbs until the next call.
'doh! Could've thought of that; makes a lot of
sense, here.
b) modify whatever block is generating
messages so that
it outputs
0x7E frames periodically
c) modify the FPGA (or whatever DUC backend you're
using) to both
do the modulating and send 0x7Es (modulated) when
there's nothing
coming in
Each approach has its disadvantages:
a) because Gnuradio buffers can be quite
large, you
will incur a
latency penalty as GR (i.e., all of the
buffers of all
of the
blocks downstream of the HDLC framer)
propagates your
frame downstream
Not if he adds the functionality to include a
tx_time tag; that
wouldn't /reduce/ latency, but it would allow for
time-slotted
transmissions!
b) message sources don't have backpressure, so
this
won't really
work -- either you'll underflow and interrupt your
sample stream
or build up a huge stack of unsent data just
as in a)
c) this is a lot of work, and requires that
the whole
modulator,
soup-to-nuts, is implemented in the FPGA.
I'd start with approach a) and work your way
towards c)
if your
application can't handle the latency.
--n
Best regards,
Marcus
_______________________________________________
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org <mailto:Discuss-gnuradio@gnu.org>
<mailto:Discuss-gnuradio@gnu.org <mailto:Discuss-gnuradio@gnu.org>>
<mailto:Discuss-gnuradio@gnu.org
<mailto:Discuss-gnuradio@gnu.org>
<mailto:Discuss-gnuradio@gnu.org <mailto:Discuss-gnuradio@gnu.org>>>
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
<https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>
<https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
<https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>>
<https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
<https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>
<https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
<https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>>>
_______________________________________________
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org <mailto:Discuss-gnuradio@gnu.org>
<mailto:Discuss-gnuradio@gnu.org <mailto:Discuss-gnuradio@gnu.org>>
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
<https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>
<https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
<https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>>
_______________________________________________
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org <mailto:Discuss-gnuradio@gnu.org>
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
<https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>
_______________________________________________
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
