Re: [Discuss-gnuradio] About closing flowgraph automatically
Hi Piotr,
> Maybe it can be some "exit" tag that would be propagated to all blocks
> in a flowgraph.
In fact, GR does have something like that: It's a built-in message
handler for a message port called "system"; see block.cc:60:
60 message_port_register_in(pmt::mp("system"));
61 set_msg_handler(pmt::mp("system"), boost::bind(&block::system_handler,
this, _1));
You can send a "done" message there, and the block will set it's
internal state to being done, ie work won't be called any more, the
stream and message neighbours of that block will be informed to be done,
too.
By the way, the information you're looking for mainly resides in
block.cc, block_executor.cc's "run_one_iteration" method, and
tpb_thread_body.cc
It's exactly the mechanism that happens when a block notifies its
message neighbours to be done; for example, let ==> be a stream
connection and --> a message connection
msg_burster ---> some_kind_of_msg_to_stream ==> multiply_const ==> head
==> file_sink
at some point, head's work method will tell the scheduler it's done. The
scheduler will then inform head's stream neighbours (because head
doesn't have any message neighbours) that they should be done, too.
So file sink closes its file, mutliply_const is told to stop, and
multiply_const then notifies its stream neighbours itself, and thus,
some_kind_of_msg_to_stream is told to stop doing its thing, too.
However, if these neigbours (which, remember, are all independently
running threads!) are currently in their work() methods, then the
scheduler won't (can't) interrupt that work call. That's all fine for
things like multiply_const, where the work() will be done as soon as it
has gone through the chunk of samples it's currently processing.
For stream sources such as some_kind_of_msg_to_stream, the problem is a
little more complex:
Stream sources are usually designed to be blocking in the work() method;
if that blocking doesn't have some timeout, then there's no way for that
block's work() to ever stop blocking, and hence, the flow graph won't
shut down, because the "system" message handler never gets called. In
the some_kind_of_msg_to_stream source, the programmer must take care not
to block in work() if there's nothing to produce, but just return 0 -
meaning the source didn't produce anything. In the olden,
pre-message-passing days, that would've been a great mistake, because a
source producing nothing although there was the whole output buffer in
space was declared finished, but nowadays, the scheduler can "revive" a
source block not only when the free space in the output changes, but
also if there's new messages to be handled. HOWEVER: this might still
have bugs, and it might be that this is what you're encountering.
Debugging your flowgraphs on a C++ and threat level when they ought to
be exiting would be pretty interesting!
Best regards,
Marcus
On 23.07.2016 08:43, Piotr Krysik wrote:
> Hi Simone and all,
>
> Can you provide your GRC flow-graph? I think that source of the problem
> might be somewhere in this part
> "PMT MESSAGE TO FILE SOURCE" as there is message to samples stream
> conversion involved here. But it's not clear for me what it actually is
> as you described it.
>
> I have problems with automatic closing of a flow-graph that uses a block
> that converts messages to samples. In my case it is PDU to Tagged Stream
> block that caused problems with automatic exiting of the flow-graph.
>
> The constant problems with exiting of flow-graphs lead me to think if
> these problems can be solved once for all with some change in gnuradio's
> internals.
>
> Maybe it can be some "exit" tag that would be propagated to all blocks
> in a flowgraph. This tag would be processed by all blocks and would
> cause them to be marked as ready for the program exit. Blocks that have
> stream inputs and message outputs would convert the tag to an "exit"
> message, which would be propagated through the outputs.
> Blocks with message inputs ans stream outputs would convert "exit"
> message to an "exit" tag. When all blocks would be marked as ready for
> closing the program would end.
>
> In this concept there is problem how to propagate the information to
> blocks upstream, that might not have information about ending of a
> flow-graph. So the method I described might be used as an addition to
> current method of ending flowgraphs.
>
> What do you think?
>
> Best Regards,
> Piotr Krysik
>
> W dniu 22.07.2016 o 18:40, Martin Braun pisze:
>> Simone,
>>
>> a file source *will* terminate the flow graph once its read the file
>> (unless you specify 'repeat').
>>
>> M
>>
>> On 07/21/2016 12:32 AM, Simone Ciccia S210664 wrote:
>>> Goodmorning,
>>> I would like to Know some methods to Close flowgraph automatically when
>>> it has finished. Some example, stop when USRP has no more samples to
>>> transmits, or a file source has read until EOF.
>>> The Run of completion option works when (for instance) you have a file
>>> source connected to other no
Re: [Discuss-gnuradio] Understanding tags_demo.cc example in gr-uhd
Stressing "average rate"! In GNU Radio, samples are processed in as much space there's output buffer and/or how many samples a block wants to consume or produce at once. In the case of throttle, a single call to its work() method takes all the input there is (let's say it's 8000 samples in this round; the value depends on how the flow graph operates, and can and will change from iteration to iteration!), copies it to the output buffer (the sync_block idiom in GNU Radio ensures that there's always at least as much output space available as input, so the min(input available, output space available) is handed to the work() function), and then just waits as long as it needs to wait: simply n_samples_copied * sampling_rate. That means that if you set the sampling rate to 1, the throttle will block execution for 8000s, and then the downstream block will be informed that there are 8000 new items to be processed. A common misconception is that setting a sampling rate would make throttle emit 1 sample every 1/sampling rate, which is not the case. This, by the way, is the reason why it's *always* bad to have a throttle block connected to some hardware sink, even if the hardware sink's sampling rate is lower than the throttle's rate; the "chunky" nature of passing samples makes it likely that the samples simply don't reach the hardware sink in time. Best regards, Marcus On 23.07.2016 03:51, Martin Braun wrote: > On 07/22/2016 04:38 PM, Lakshay Narula wrote: >> Thanks Martin, that was exactly what I was looking for. >> >> Would the following statement be correct: the GNU Radio flowgraph always >> produces samples as fast as the CPU can (irrespective of the set sample >> rate or throttle), and stops producing whenever the USRP sink buffer is >> full. > Yup. > >> Also, does the throttle block work in a similar fashion? > That block blocks work() such that the average rate through it is what > it's set to to. > > M >> Thanks again, >> Lakshay. > > ___ > Discuss-gnuradio mailing list > Discuss-gnuradio@gnu.org > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] About closing flowgraph automatically
Hi Marcus,
>Debugging your flowgraphs on a C++ and threat level when they ought to
>be exiting would be pretty interesting!
An example of my flowgraph where I used PDU to tagged stream convertion
is this:
https://github.com/ptrkrysik/examples/blob/master/voice_decoding_in_grc/tch_f_reception.grc
Messages with GSM coded voice frames are turned into stream so they can
be decoded with GSM full-rate audio decoder and played in gnuradio audio
sink.
This might be too big flow-graph for the purpose of debugging, so I
prepared minimal working example presenting the problem. The flowgraph
and its input file are attached to this post. If you want to run it in
GRC you will have to correct the File option of the file meta source to
absolute path to the tagged_stream file. I don't know how to do relative
paths in GRC yet.
Best Regards,
Piotr
W dniu 23.07.2016 o 11:55, Marcus Müller pisze:
> Hi Piotr,
>
>> Maybe it can be some "exit" tag that would be propagated to all blocks
>> in a flowgraph.
> In fact, GR does have something like that: It's a built-in message
> handler for a message port called "system"; see block.cc:60:
>
> 60 message_port_register_in(pmt::mp("system"));
> 61 set_msg_handler(pmt::mp("system"),
> boost::bind(&block::system_handler, this, _1));
>
> You can send a "done" message there, and the block will set it's
> internal state to being done, ie work won't be called any more, the
> stream and message neighbours of that block will be informed to be done,
> too.
>
> By the way, the information you're looking for mainly resides in
> block.cc, block_executor.cc's "run_one_iteration" method, and
> tpb_thread_body.cc
>
> It's exactly the mechanism that happens when a block notifies its
> message neighbours to be done; for example, let ==> be a stream
> connection and --> a message connection
>
> msg_burster ---> some_kind_of_msg_to_stream ==> multiply_const ==> head
> ==> file_sink
>
> at some point, head's work method will tell the scheduler it's done. The
> scheduler will then inform head's stream neighbours (because head
> doesn't have any message neighbours) that they should be done, too.
>
> So file sink closes its file, mutliply_const is told to stop, and
> multiply_const then notifies its stream neighbours itself, and thus,
> some_kind_of_msg_to_stream is told to stop doing its thing, too.
>
> However, if these neigbours (which, remember, are all independently
> running threads!) are currently in their work() methods, then the
> scheduler won't (can't) interrupt that work call. That's all fine for
> things like multiply_const, where the work() will be done as soon as it
> has gone through the chunk of samples it's currently processing.
>
> For stream sources such as some_kind_of_msg_to_stream, the problem is a
> little more complex:
>
> Stream sources are usually designed to be blocking in the work() method;
> if that blocking doesn't have some timeout, then there's no way for that
> block's work() to ever stop blocking, and hence, the flow graph won't
> shut down, because the "system" message handler never gets called. In
> the some_kind_of_msg_to_stream source, the programmer must take care not
> to block in work() if there's nothing to produce, but just return 0 -
> meaning the source didn't produce anything. In the olden,
> pre-message-passing days, that would've been a great mistake, because a
> source producing nothing although there was the whole output buffer in
> space was declared finished, but nowadays, the scheduler can "revive" a
> source block not only when the free space in the output changes, but
> also if there's new messages to be handled. HOWEVER: this might still
> have bugs, and it might be that this is what you're encountering.
> Debugging your flowgraphs on a C++ and threat level when they ought to
> be exiting would be pretty interesting!
>
> Best regards,
>
> Marcus
>
> On 23.07.2016 08:43, Piotr Krysik wrote:
>> Hi Simone and all,
>>
>> Can you provide your GRC flow-graph? I think that source of the problem
>> might be somewhere in this part
>> "PMT MESSAGE TO FILE SOURCE" as there is message to samples stream
>> conversion involved here. But it's not clear for me what it actually is
>> as you described it.
>>
>> I have problems with automatic closing of a flow-graph that uses a block
>> that converts messages to samples. In my case it is PDU to Tagged Stream
>> block that caused problems with automatic exiting of the flow-graph.
>>
>> The constant problems with exiting of flow-graphs lead me to think if
>> these problems can be solved once for all with some change in gnuradio's
>> internals.
>>
>> Maybe it can be some "exit" tag that would be propagated to all blocks
>> in a flowgraph. This tag would be processed by all blocks and would
>> cause them to be marked as ready for the program exit. Blocks that have
>> stream inputs and message outputs would convert the tag to an "exit"
>> message, which would be propagated through the outputs.
>> Blocks with m
Re: [Discuss-gnuradio] About closing flowgraph automatically
Hi Piotr,
I let gdb loose on your minimal example [1], and found this:
52 int pdu_to_tagged_stream_impl::calculate_output_stream_length(const
gr_vector_int &)
53 {
54 if (d_curr_len == 0) {
55 /* FIXME: This blocking call is far from ideal but is the best we
56*can do at the moment
57*/
58 pmt::pmt_t msg(delete_head_blocking(PDU_PORT_ID, 100));
59 if (msg.get() == NULL) {
60 return 0;
61 }
Sylvain, who replaced the delete_head_nowait with the
delete_head_blocking call correctly pointed out the reason to do so in
his commit message(838793ce):
> blocks: Fix pdu_to_tagged_stream CPU spinning using blocking with timeout
>
> This is not ideal, but until the scheduler supports something better, this
> will have to do.
Problem is that if we use the non-blocking call here, the scheduler
would have a chance to process the shutdown signal, but it would be
constantly asking (spinning) for the output stream length.
You could try out what would happen if we'd added a timeout to the
blocking cal; that way, you could reduce the spinning, and hopefully get
the scheduler to check for "done" messages. For tagged stream blocks, my
knowledge of the scheduler control flow isn't as reliable as for regular
blocks :/
Best regards,
Marcus
[1] http://marcus.hostalia.de/screencast/run_to_completion.html
On 23.07.2016 19:41, Piotr Krysik wrote:
> Hi Marcus,
>
>> Debugging your flowgraphs on a C++ and threat level when they ought to
>> be exiting would be pretty interesting!
> An example of my flowgraph where I used PDU to tagged stream convertion
> is this:
> https://github.com/ptrkrysik/examples/blob/master/voice_decoding_in_grc/tch_f_reception.grc
>
> Messages with GSM coded voice frames are turned into stream so they can
> be decoded with GSM full-rate audio decoder and played in gnuradio audio
> sink.
>
> This might be too big flow-graph for the purpose of debugging, so I
> prepared minimal working example presenting the problem. The flowgraph
> and its input file are attached to this post. If you want to run it in
> GRC you will have to correct the File option of the file meta source to
> absolute path to the tagged_stream file. I don't know how to do relative
> paths in GRC yet.
>
> Best Regards,
> Piotr
>
>
> W dniu 23.07.2016 o 11:55, Marcus Müller pisze:
>> Hi Piotr,
>>
>>> Maybe it can be some "exit" tag that would be propagated to all blocks
>>> in a flowgraph.
>> In fact, GR does have something like that: It's a built-in message
>> handler for a message port called "system"; see block.cc:60:
>>
>> 60 message_port_register_in(pmt::mp("system"));
>> 61 set_msg_handler(pmt::mp("system"),
>> boost::bind(&block::system_handler, this, _1));
>>
>> You can send a "done" message there, and the block will set it's
>> internal state to being done, ie work won't be called any more, the
>> stream and message neighbours of that block will be informed to be done,
>> too.
>>
>> By the way, the information you're looking for mainly resides in
>> block.cc, block_executor.cc's "run_one_iteration" method, and
>> tpb_thread_body.cc
>>
>> It's exactly the mechanism that happens when a block notifies its
>> message neighbours to be done; for example, let ==> be a stream
>> connection and --> a message connection
>>
>> msg_burster ---> some_kind_of_msg_to_stream ==> multiply_const ==> head
>> ==> file_sink
>>
>> at some point, head's work method will tell the scheduler it's done. The
>> scheduler will then inform head's stream neighbours (because head
>> doesn't have any message neighbours) that they should be done, too.
>>
>> So file sink closes its file, mutliply_const is told to stop, and
>> multiply_const then notifies its stream neighbours itself, and thus,
>> some_kind_of_msg_to_stream is told to stop doing its thing, too.
>>
>> However, if these neigbours (which, remember, are all independently
>> running threads!) are currently in their work() methods, then the
>> scheduler won't (can't) interrupt that work call. That's all fine for
>> things like multiply_const, where the work() will be done as soon as it
>> has gone through the chunk of samples it's currently processing.
>>
>> For stream sources such as some_kind_of_msg_to_stream, the problem is a
>> little more complex:
>>
>> Stream sources are usually designed to be blocking in the work() method;
>> if that blocking doesn't have some timeout, then there's no way for that
>> block's work() to ever stop blocking, and hence, the flow graph won't
>> shut down, because the "system" message handler never gets called. In
>> the some_kind_of_msg_to_stream source, the programmer must take care not
>> to block in work() if there's nothing to produce, but just return 0 -
>> meaning the source didn't produce anything. In the olden,
>> pre-message-passing days, that would've been a great mistake, because a
>> source producing nothing although there was the whole output buffer
Re: [Discuss-gnuradio] Understanding tags_demo.cc example in gr-uhd
Alright. Thanks for your help Martin. -Lakshay. On Fri, Jul 22, 2016 at 8:51 PM, Martin Braun wrote: > On 07/22/2016 04:38 PM, Lakshay Narula wrote: > > Thanks Martin, that was exactly what I was looking for. > > > > Would the following statement be correct: the GNU Radio flowgraph always > > produces samples as fast as the CPU can (irrespective of the set sample > > rate or throttle), and stops producing whenever the USRP sink buffer is > > full. > > Yup. > > > Also, does the throttle block work in a similar fashion? > > That block blocks work() such that the average rate through it is what > it's set to to. > > M > > > > Thanks again, > > Lakshay. > > ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] Understanding tags_demo.cc example in gr-uhd
That was great information Marcus, appreciate it! Thanks, Lakshay. ___ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
Re: [Discuss-gnuradio] About closing flowgraph automatically
Hi,
> 52 int pdu_to_tagged_stream_impl::calculate_output_stream_length(const
> gr_vector_int &)
> 53 {
> 54 if (d_curr_len == 0) {
> 55 /* FIXME: This blocking call is far from ideal but is the best
> we
> 56*can do at the moment
> 57*/
> 58 pmt::pmt_t msg(delete_head_blocking(PDU_PORT_ID, 100));
> 59 if (msg.get() == NULL) {
> 60 return 0;
> 61 }
[snip]
> Problem is that if we use the non-blocking call here, the scheduler would
> have a chance to process the shutdown signal, but it would be constantly
> asking (spinning) for the output stream length.
>
> You could try out what would happen if we'd added a timeout to the blocking
> cal; that way, you could reduce the spinning, and hopefully get the scheduler
> to check for "done" messages.
There _is_ a timeout ... that "100" in there is the # of millisec to
wait at most.
Cheers,
Sylvain
___
Discuss-gnuradio mailing list
Discuss-gnuradio@gnu.org
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
