On Tue, 28 May 2019 10:41:03 -0700 (PDT) Steven Estes <est...@yottadb.com> wrote:
> ... anybody else reading this) have any other thoughts on what I can try > since GODEBUG="cgocheck=2" did not show anything? Nor (in summary) > has -msan, -race or the memory sanity debugging built into our C code. > We're kinda stuck at this point. On Wed, 22 May 2019 15:16:11 -0700 (PDT) Steven Estes <est...@yottadb.com> wrote: > With the Golang wrapper, we have run into a variety of panics and crashes > that take several forms. Those forms in order of frequency are as follows: > - Panics due to “sweep increased allocation count”. > - Panics due to invalid address (bad pointer) in Go heap. > - SIG-11 failures in various places usually dealing with garbage > collection. > - Panics due to “unknown return address” when trying to return from > functions. > - Mutex deadlocks (everyone waiting and nobody holding) Its a telltale of corrupt memory (I haven't yet seen Go corrupting C side). Likely C code happily writes to the address that was freed at Go space. > Running with either of the options “-msan” or “-race” has not Neither can help with C process gone wild. It is the Go code instrumented, and msan just informs common (C/LLVM) sanitizer about Go writes and reads (both of Go code usually does within its own boundaries). > A bit of background: > [...] > we make use of Finalizers when the C storage is allocated so when the base > Golang structure is garbage collected, a function is run that releases the > allocated C memory attached to it. So far so good. No. It is not good. Please read carefully: https://golang.org/pkg/runtime/#SetFinalizer paying high attention to phrases starting with "It is not guaranteed..." > The C API that we are wrapping has parameters that are pointers to > structures that themselves contain pointers to information... [...] hundreds of lines of description from OP cut [...] > Anyway, any insights as to how to make progress with this issue would be > most appreciated. I’ve tried to keep this as brief as possible. >From this brief I understood it all was supposed to be "super fast" but after reading https://docs.yottadb.com/MultiLangProgGuide/cprogram.html I am not so sure. So I can not offer anything above Michael Jones' advice: **separate memory concerns with shims**: >MJ Michael Jones <michael.jo...@gmail.com> wrote: >MJ and when you change that "pass a pointer" to "have a shim c-side function >MJ return a uint32 that you just assign on the Go side > We don’t understand what the Golang errors are telling us. They tells a tale of corrupt memory. C side probably is writing to the locations Go already moved, freed and even assigned to some other data structure. [Less probable but still possible is corruption due to "unsafe" fiddling made wrong.] You apparently have assumed that if something is called Finalizer it will run. It more often will not (and not only in Go[1]). Because of that, alas, I'd suggest to rethink architecture then start anew. (Shredding all code made on wrong assumptions and starting it anew is way cheaper man-hour wise than refurbishing then debugging for leftovers.) [1] https://hownot2code.com/2016/10/07/why-using-finalizers-is-a-bad-idea/ > Steve Hope this helps. -- Wojciech S. Czarnecki << ^oo^ >> OHIR-RIPE -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/20190528225513.56a014e4%40zuzia. For more options, visit https://groups.google.com/d/optout.
