Re: double free or corruption error when two instances of an OOT block using fftw is called in GRC

Fri, 21 Apr 2023 13:21:41 -0700 

Hi Grace,
as said, this code shows that you don't understand how long (or rather: short) the object `plans` lives! This is a basic C++ issue: things created in local scope of a function end to exist at the end of the function. (I honestly think you'll need a bit of a better introduction to C++ – this will be easier for you with a solid understanding!) 
So, your code needs fundamental rework:


- You currently always have to make a new FFTW plan, because your input might always be at 
a new address. That's bad. Instead, you must always put the samples into the same input 
buffer of your FFT. Then you can reuse the same plan over and over again.
- That means you have to rewrite ApplyFilterBank, so that it always puts the samples into 
the input buffer of your FFT.
- As soon as you've done that, you can use `gr::fft::fft<float, true>` (which gives you an 
FFTw of type float that is forward=true), instead of doing **anything** with FFTW yourself.

- You can then just throw away your `ForwardFT` function. Instead,
  - add a `std::unordered_map<unsigned int, gr::fft::fft<float, true>> 
prepared_ffts` to
    your class's private members in probeExtract_impl.h, then
  - in `ApplyFilterBank`, you just do something like

    auto& [fft, already_there] = prepared_ffts.try_emplace(M, M, 
number_of_threads);
    auto& input_buffer = fft.get_inbuf();
    // TODO: put all input into input_buffer
    fft.execute()
    // output is now in fft.get_outbuf()


That's it. You don't have to care about any fftwf_init_threads, about any _delete_plan, 
you don't need to create the plans yourself.

Best,
Marcus

On 21.04.23 17:53, Grace Yeung wrote:

Hi Marcus,


Here is the ForwardFT() code below that is called by ApplyFilterBank() in the work 
function. So does it mean I should compute the plan only once in the constructor and use 
it throughout the work function by passing it to the ForwardFT() funciton? And declare 
_plans in the impl.h file?  Thanks.

Grace

       void probeExtract_impl::ForwardFT (vector<gr_complex> &series, int M)
       {
           int exponent;
           fftwf_plan plans[MAX_FFT_EXP];

           exponent = (int) (log10((float) M)/log10(2.0) + 0.5);

           err_val = fftwf_init_threads ();
           if (err_val == 0)
               throw runtime_error("fftwf_init_threads failed");
           fftwf_plan_with_nthreads(NUM_FFTW_THREADS);

           pthread_mutex_lock (&fftw3_mutex);

           fftwf_make_planner_thread_safe();

           plans[exponent] = fftwf_plan_dft_1d (M,
reinterpret_cast<fftwf_complex *> (&series[0]),
reinterpret_cast<fftwf_complex *> (&series[0]),
                                                FFTW_FORWARD, FFTW_ESTIMATE);
           fftwf_execute (plans[exponent]);

           pthread_mutex_unlock (&fftw3_mutex);
           fftwf_destroy_plan (plans[exponent]);
           fftwf_cleanup_threads ();

           // Notes: destroying plans/cleaning up threads appear to eliminate growing 
memory issue 2/2023

           return;
       }


On 4/21/23 08:20, Grace Yeung wrote:

Thank you Marcus for taking the time to look through the backtrace error. 
ApplyFilterBank simply calls ForwardFT to perform the FFT and does not access the plan. 
It only uses the FFT output. I had to destroy the plan each time because I noticed 
memory was consistently growing when gnuradio was running the flowgraph. After adding 
the destroy_plan memory stops growing. Hope this makes sense. Thank you.

Grace

—​
Grace K. Yeung, MS
NorthWest Research Associates
301 Webster Street
Monterey, CA 93940
gr...@nwra.com
https://www.nwra.com
------------------------------------------------------------------------------------------
*From:* Marcus Müller <mmuel...@gnuradio.org>
*Sent:* Friday, April 21, 2023 7:10:43 AM
*To:* Grace Yeung <gr...@nwra.com>

*Subject:* Re: double free or corruption error when two instances of an OOT block using 
fftw is called in GRC

Hi Grace,

reading this backtrace, the error happens in

#29 0x00007fffe99b04d5 in gr::customModule::probeExtract_impl::ApplyFilterBank

So, that's where you need to look, not sure whether the other code you shared 
helps.

But reading your other code: you seem to be making an FFT plan and destroying 
it *every
single time you do an FFT*. That's a terrible idea – planning an FFT typically 
takes much
longer than doing an FFT, even a really badly implemented one.
You also make a local array of plans, just to put that single plan in, then 
destroy it.
This indicates you've got a logical bug in your thinking about C++ scoping and 
lifetimes.
If I had to take a wild guess, you're trying to access a plan that does no 
longer exist in
your ApplyFilterBank.

Best regards,
Marcus

On 19.04.23 20:18, Grace Yeung wrote:

> After some debugging, it appears grc is breaking inside of calls to fftw (see below). 
I am

> simply trying to use fftw to perform a simple FFT transform 
(forward/backward). Is there
> anything special I need to watch out for when using fftw with gnuradio? Any 
examples
> available? Any advice would be much appreciated. Thanks!
>
> Backtracing shows the crash happens in an fftw3f library function used in the 
code...
>
> (gdb) bt
> #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
> #1 <https://github.com/gnuradio/gnuradio/pull/1>0x00007ffff7dd1859 in 
__GI_abort () at
> abort.c:79
> #2 <https://github.com/gnuradio/gnuradio/pull/2>0x00007ffff7e3c26e in 
__libc_message
> (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff7f66298 "%s\n") at
> ../sysdeps/posix/libc_fatal.c:155
> #3 <https://github.com/gnuradio/gnuradio/pull/3>0x00007ffff7e442fc in 
malloc_printerr
> (str=str@entry=0x7ffff7f68628 "double free or corruption (fasttop)") at 
malloc.c:5347
> #4 <https://github.com/gnuradio/gnuradio/pull/4>0x00007ffff7e45c65 in 
_int_free
> (av=0x7fffac000020, p=0x7fffac55a8d0, have_lock=0) at malloc.c:4266
> #5 <https://github.com/gnuradio/gnuradio/pull/5>0x00007fffefe830d9 in ?? () 
from
> /lib/x86_64-linux-gnu/libfftw3f.so.3
> #6 <https://github.com/gnuradio/gnuradio/pull/6>0x00007fffefe83c44 in ?? () 
from
> /lib/x86_64-linux-gnu/libfftw3f.so.3
> #7 <https://github.com/gnuradio/gnuradio/pull/7>0x00007fffefe83fc4 in 
fftwf_mkplan_d ()
> from /lib/x86_64-linux-gnu/libfftw3f.so.3
> #8 <https://github.com/gnuradio/gnuradio/pull/8>0x00007fffefe88e5d in ?? () 
from
> /lib/x86_64-linux-gnu/libfftw3f.so.3
> #9 <https://github.com/gnuradio/gnuradio/pull/9>0x00007fffefe8380b in ?? () 
from
> /lib/x86_64-linux-gnu/libfftw3f.so.3
> #10 <https://github.com/gnuradio/gnuradio/pull/10>0x00007fffefe83a03 in ?? () 
from
> /lib/x86_64-linux-gnu/libfftw3f.so.3
> #11 <https://github.com/gnuradio/gnuradio/pull/11>0x00007fffefe83fc4 in 
fftwf_mkplan_d ()
> from /lib/x86_64-linux-gnu/libfftw3f.so.3
> #12 <https://github.com/gnuradio/gnuradio/pull/12>0x00007fffefe8d95c in ?? () 
from
> /lib/x86_64-linux-gnu/libfftw3f.so.3
> #13 <https://github.com/gnuradio/gnuradio/pull/13>0x00007fffefe8380b in ?? () 
from
> /lib/x86_64-linux-gnu/libfftw3f.so.3
> #14 <https://github.com/gnuradio/gnuradio/pull/14>0x00007fffefe83a03 in ?? () 
from
> /lib/x86_64-linux-gnu/libfftw3f.so.3
> #15 <https://github.com/gnuradio/gnuradio/pull/15>0x00007fffefe83fc4 in 
fftwf_mkplan_d ()
> from /lib/x86_64-linux-gnu/libfftw3f.so.3
> #16 <https://github.com/gnuradio/gnuradio/pull/16>0x00007fffefe84043 in 
fftwf_mkplan_f_d
> () from /lib/x86_64-linux-gnu/libfftw3f.so.3
> #17 <https://github.com/gnuradio/gnuradio/pull/17>0x00007fffefe887a3 in ?? () 
from
> /lib/x86_64-linux-gnu/libfftw3f.so.3
> #18 <https://github.com/gnuradio/gnuradio/pull/18>0x00007fffefe8380b in ?? () 
from
> /lib/x86_64-linux-gnu/libfftw3f.so.3
> #19 <https://github.com/gnuradio/gnuradio/pull/19>0x00007fffefe83a03 in ?? () 
from
> /lib/x86_64-linux-gnu/libfftw3f.so.3
> #20 <https://github.com/gnuradio/gnuradio/pull/20>0x00007fffefe83fc4 in 
fftwf_mkplan_d ()
> from /lib/x86_64-linux-gnu/libfftw3f.so.3
> #21 <https://github.com/gnuradio/gnuradio/pull/21>0x00007fffefe88e5d in ?? () 
from
> /lib/x86_64-linux-gnu/libfftw3f.so.3
> #22 <https://github.com/gnuradio/gnuradio/pull/22>0x00007fffefe8380b in ?? () 
from
> /lib/x86_64-linux-gnu/libfftw3f.so.3
> #23 <https://github.com/gnuradio/gnuradio/pull/23>0x00007fffefe83a03 in ?? () 
from
> /lib/x86_64-linux-gnu/libfftw3f.so.3
> #24 <https://github.com/gnuradio/gnuradio/pull/24>0x00007fffeff4a583 in ?? () 
from
> /lib/x86_64-linux-gnu/libfftw3f.so.3

> #25 <https://github.com/gnuradio/gnuradio/pull/25>0x00007fffeff4a758 in 
fftwf_mkapiplan ()

> from /lib/x86_64-linux-gnu/libfftw3f.so.3
> #26 <https://github.com/gnuradio/gnuradio/pull/26>0x00007fffeff4ddd7 in
> fftwf_plan_many_dft () from /lib/x86_64-linux-gnu/libfftw3f.so.3
> #27 <https://github.com/gnuradio/gnuradio/pull/27>0x00007fffeff4d09b in 
fftwf_plan_dft ()
> from /lib/x86_64-linux-gnu/libfftw3f.so.3
> #28 <https://github.com/gnuradio/gnuradio/pull/28>0x00007fffeff4cd9a in 
fftwf_plan_dft_1d
> () from /lib/x86_64-linux-gnu/libfftw3f.so.3
> #29 <https://github.com/gnuradio/gnuradio/pull/29>0x00007fffe99b04d5 in
> gr::customModule::probeExtract_impl::ApplyFilterBank(int,
> gr::customModule::probeExtract_impl::FilterBank, std::vector<std::complex,
> std::allocator<std::complex > >&, int, int, int, std::vector<float, std::allocator 
>&,
> int, int, int) ()
> from /usr/local/lib/x86_64-linux-gnu/libgnuradio-customModule.so.1.0.0git
> #30 <https://github.com/gnuradio/gnuradio/pull/30>0x00007fffe99b1f69 in

> gr::customModule::probeExtract_impl::general_work(int, std::vector<int, std::allocator 
>&,

> std::vector<void const*, std::allocator<void const*> >&, std::vector<void*,
> std::allocator<void*> >&) () from
> /usr/local/lib/x86_64-linux-gnu/libgnuradio-customModule.so.1.0.0git
> #31 <https://github.com/gnuradio/gnuradio/pull/31>0x00007ffff0799c03 in
> gr::block_executor::run_one_iteration() () from
> /lib/x86_64-linux-gnu/libgnuradio-runtime.so.3.8.1
> #32 <https://github.com/gnuradio/gnuradio/pull/32>0x00007ffff07ed59a in
> gr::tpb_thread_body::tpb_thread_body(boost::shared_ptrgr::block,
> boost::shared_ptrboost::barrier, int) ()
> from /lib/x86_64-linux-gnu/libgnuradio-runtime.so.3.8.1
> #33 <https://github.com/gnuradio/gnuradio/pull/33>0x00007ffff07dd818 in ?? () 
from
> /lib/x86_64-linux-gnu/libgnuradio-runtime.so.3.8.1
> #34 <https://github.com/gnuradio/gnuradio/pull/34>0x00007ffff07fc4e8 in ?? () 
from
> /lib/x86_64-linux-gnu/libgnuradio-runtime.so.3.8.1
> #35 <https://github.com/gnuradio/gnuradio/pull/35>0x00007ffff010343b in ?? () 
from
> /lib/x86_64-linux-gnu/libboost_thread.so.1.71.0
> #36 <https://github.com/gnuradio/gnuradio/issues/36>0x00007ffff7d94609 in 
start_thread
> (arg=) at pthread_create.c:477
> #37 <https://github.com/gnuradio/gnuradio/pull/37>0x00007ffff7ece133 in clone 
() at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
>
>
>
>
> On 2/15/23 13:01, Grace Yeung wrote:
>> Hi, I have a gnuradio out-of-tree block written in C++ that runs fine in grc 
3.8.1.0.
>> But when I have two of them running simultaneously in the flowgraph, grc 
crashes and
>> gives me the error:
>>
>> double free or corruption (!prev)
>>
>> I am not using any free() functions. I use std::vector's in my code which 
should handle
>> dynamic memory allocation automatically. What could be causing this error?
>>
>> The only other place I see in the code that deals with pointers is the calling of 
"new"
>> in the definition of the make function which returns an sptr:
>>       probeExtract::sptr
>>       probeExtract::make(const char* filtName, const char* maskName, int 
mlsrM, int
>> numChan, int sampPerBit, int Tblock, int edgeBufferSize, int structureType)
>>       {
>>           return gnuradio::get_initial_sptr
>>           (new probeExtract_impl(filtName, maskName, mlsrM, numChan, 
sampPerBit, Tblock,
>> edgeBufferSize, structureType));
>>       }
>> I suppose I do not need to worry about free'ing the pointer which is handled 
internally
>> by python?
>>
>> After putting in some debugging statements, I see variations of similar 
error such as:
>> corrupted size vs. prev_size while consolidating
>>
>> Any advice would be greatly appreciated. Thank you.
>>
>> Grace
>>
>
> --
> Grace K. Yeung, MS
> NorthWest Research Associates
> 301 Webster Street
> Monterey, CA 93940
> gr...@nwra.com
> https://www.nwra.com
>

--
Grace K. Yeung, MS
NorthWest Research Associates
301 Webster Street
Monterey, CA 93940
gr...@nwra.com
https://www.nwra.com























					Reply via email to