It is really impossible to help you debug what is to the thousands of
readers here an invisible unseen program. You have a logic error. You don't
know where it is (of course, you are busy trying to find it) but we can't
know either as we don't see the code. General advice, as always, is to
"trap" the bug in the smallest stand-alone code that you can. That will
focus your search.
You just shared this:
Following is one of the methods called before this method :
func (obj SyslogParser) InitializeEngine() {
patterndb := C.CString(obj.Patterndb)
defer C.free(unsafe.Pointer(patterndb))
modulepath := C.CString(obj.Modulepath)
defer C.free(unsafe.Pointer(modulepath))
C.initialize_engine(patterndb, modulepath)
}
Not deallocating the memory for the struct can be the issue for 'fatal
error: bin/main': double free or corruption (fasttop)' ?
Is this right? Is it correct to call C.initialize_engine(a,b) then
immediately call C.free(a) and C.free(b)? It might be fine if your
initialize engine clones/duplicates/or otherwise uses a & b just once. But
if not, then of course A & B will both instantly become death traps for
your C code during runtime of the engine, for any allocations that are
where A & B were located, and potentially when your "engine" is deallocated
later on.
Just one example of might be fine, might be disaster, no way for us to
tell. It seems that you have a logic problem in the program, not a C-to-Go
or Go-to-C interfacing problem. You're going to need to study your code
(unseen by us) yourself to see what that is. I think so anyway. Maybe Ian
is so smart that he can debug this for you by ESP. ;-)
Michael
On Fri, Jul 12, 2019 at 8:15 AM Nitish Saboo <nitish.sabo...@gmail.com>
wrote:
> Hi Ian,
>
> > syslogparser.go
> > =============
> >
> > func (obj SyslogParser) LoadPatternDB(opts Syslog, workerId int) {
> log.Info("Loading pattern db with path:", opts.Patterndb)
> > patterndbpath := C.CString(opts.Patterndb) . <<<<<<<<<<<<<<< . STEP 1
> > defer C.free(unsafe.Pointer(patterndbpath)) . <<<<<<<<<<<<<<<< STEP 2
> > InitStruct := (*C.Accumulatedparams)(C.calloc(1,
> C.sizeof_struct_Accumulatedparams))
> > InitStruct.callback = (C.key_value_cb)(C.callback)
> > InitStruct.data = C.int(workerId)
> > C.load_pattern_db(patterndbpath,
> (*C.Accumulatedparams)(unsafe.Pointer(InitStruct)), C.int(workerId))
> > }
> >
> > Since I am running this program in multiple go routines, when we do
> above STEP 1 :
> >
> > 1)Will the C memory allocate different addresses for 'patterndbpath' in
> both the routines ...Am I right ?
>
> Yes.
>
> > 2)You said 'It may be calling free twice on the same pointer', but I am
> assigning a different pointer in STEP 1 everytime for each routine...Am I
> right ?
>
> Yes, you will see a different pointer each time you call C.calloc, and
> in the code that I see that pointer is only freed once, but I have no
> idea what the rest of your code is doing.
>
> I am calling free only once and that is for 'patterndbpath' in this
> particular method.In the entire codebase I am NOT calling free for
> InitStruct (Memory allocated for the struct)
> I am pointing out this particular method 'LoadPatternDB' of Go code base
> because I had put a 'Info' log in this method at the starting and that got
> printed before the corruption error came up.
> And my testing failed at the third iteration:
>
> 1)WorkerID0 completed the work
> 2)WorkerID1 completed the work
> 3)WorkerID1 failed in this particular method with 'fatal error:
> bin/main': double free or corruption (fasttop)'
>
> Following is one of the methods called before this method :
>
> func (obj SyslogParser) InitializeEngine() {
> patterndb := C.CString(obj.Patterndb)
> defer C.free(unsafe.Pointer(patterndb))
> modulepath := C.CString(obj.Modulepath)
> defer C.free(unsafe.Pointer(modulepath))
> C.initialize_engine(patterndb, modulepath)
> }
>
> Not deallocating the memory for the struct can be the issue for 'fatal
> error: bin/main': double free or corruption (fasttop)' ?
>
> > 3)Is it possible that one Go routine has already freed up the address
> that the other Go routine is trying to free and that is causing this
> 'fatal error: bin/main': double free or corruption (fasttop)'
>
> Yes, that is possible. It won't happen with the code I see above but
> I have no idea what the rest of your code is doing.
>
> How can it not happen with this code ?I did not understand this point.
> Can you please clarify this point ?
>
> > Is this because of the race condition between two routines that this
> error is seen ?
>
> It is possible.
>
> > 4)If you show us a small, complete, self-contained example, that does
> not call any other code, then we may be able to see the problem.
> > >> As you have mentioned problem is happening when we call 'C.free'.Do
> you mean by not calling internal syslog-ng methods and just calling a
> normal C program with multiple go routines ?
> > Please correct me here if I am wrong
>
> I mean any small, self-contained example where we can see all the code.
>
> >>>I will try to send a very small, basic code but not sure how to
> reproduce the issue with that.
>
> You seem to be looking at a memory corruption or double free problem.
> These are normal, ordinary, problems when writing C code. We can't
> debug your problem for you.
>
> On Fri, Jul 12, 2019 at 7:54 PM Ian Lance Taylor <i...@golang.org> wrote:
>
>> On Thu, Jul 11, 2019 at 11:41 PM Nitish Saboo <nitish.sabo...@gmail.com>
>> wrote:
>> >
>> > syslogparser.go
>> > =============
>> >
>> > func (obj SyslogParser) LoadPatternDB(opts Syslog, workerId int) {
>> > patterndbpath := C.CString(opts.Patterndb) . <<<<<<<<<<<<<<< . STEP 1
>> > defer C.free(unsafe.Pointer(patterndbpath)) . <<<<<<<<<<<<<<<< STEP 2
>> > InitStruct := (*C.Accumulatedparams)(C.calloc(1,
>> C.sizeof_struct_Accumulatedparams))
>> > InitStruct.callback = (C.key_value_cb)(C.callback)
>> > InitStruct.data = C.int(workerId)
>> > C.load_pattern_db(patterndbpath,
>> (*C.Accumulatedparams)(unsafe.Pointer(InitStruct)), C.int(workerId))
>> > }
>> >
>> > Since I am running this program in multiple go routines, when we do
>> above STEP 1 :
>> >
>> > 1)Will the C memory allocate different addresses for 'patterndbpath'
>> in both the routines ...Am I right ?
>>
>> Yes.
>>
>> > 2)You said 'It may be calling free twice on the same pointer', but I am
>> assigning a different pointer in STEP 1 everytime for each routine...Am I
>> right ?
>>
>> Yes, you will see a different pointer each time you call C.calloc, and
>> in the code that I see that pointer is only freed once, but I have no
>> idea what the rest of your code is doing.
>>
>> > 3)Is it possible that one Go routine has already freed up the address
>> that the other Go routine is trying to free and that is causing this
>> 'fatal error: bin/main': double free or corruption (fasttop)'
>>
>> Yes, that is possible. It won't happen with the code I see above but
>> I have no idea what the rest of your code is doing.
>>
>> > Is this because of the race condition between two routines that this
>> error is seen ?
>>
>> It is possible.
>>
>> > 4)If you show us a small, complete, self-contained example, that does
>> not call any other code, then we may be able to see the problem.
>> > >> As you have mentioned problem is happening when we call 'C.free'.Do
>> you mean by not calling internal syslog-ng methods and just calling a
>> normal C program with multiple go routines ?
>> > Please correct me here if I am wrong
>>
>> I mean any small, self-contained example where we can see all the code.
>>
>> You seem to be looking at a memory corruption or double free problem.
>> These are normal, ordinary, problems when writing C code. We can't
>> debug your problem for you.
>>
>> Ian
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CALjMrq6eM1_ykJYmNVYqnFnjxqRvRCJF8Bqgh6k2N7kBV%2B_3Dw%40mail.gmail.com
> <https://groups.google.com/d/msgid/golang-nuts/CALjMrq6eM1_ykJYmNVYqnFnjxqRvRCJF8Bqgh6k2N7kBV%2B_3Dw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
*Michael T. jonesmichael.jo...@gmail.com <michael.jo...@gmail.com>*
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CALoEmQz-_krR6zCa5U6gWL7ZAWXcrK8brTOQQicxL5zwxF%3DYfA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
