Hmm, is it actually a performance win to offload memory-zeroing to another thread? I would think moving the cache lines between the two cores would cost more than the zeroing itself.
-Kenton On Wed, Aug 21, 2019 at 11:05 AM Vitali Lovich <[email protected]> wrote: > Oh. I'm very wrong. The check is in MallocMessageBuilder. I'll try my > custom builder. > > On Wed, Aug 21, 2019 at 11:03 AM Vitali Lovich <[email protected]> wrote: > >> I have not. I thought MessageBuilder constructor requires zero'ed memory >> so this kind of split construction to "resume" the arena isn't possible >> with a custom subclass but maybe I'm wrong? >> >> And yes I agree that I need to zero that memory but I do that on a >> background low priority thread after I serialize (the goal is to offload as >> much as possible and capture/write only the unique data that's >> irreplaceable at the tracepoint). For example I even avoid capturing the >> TID at the tracepoint because that's implicitly available when I submit >> that threads events for serialization. >> >> On Wed, Aug 21, 2019 at 10:46 AM Kenton Varda <[email protected]> >> wrote: >> >>> Hi Vitali, >>> >>> Instead of using MallocMessageBuilder, have you tried writing a custom >>> subclass of MessageBuilder? It should allow you to customize these things >>> more precisely. >>> >>> Note that *someone* has to zero out memory before Cap'n Proto can start >>> allocating from it (because Cap'n Proto structures all expect to be >>> zero-initialized anyway, so as an optimizaiton we assume memory is zero'd >>> in advance), but with a custom MessageBuilder you have more control over >>> when it happens. >>> >>> -Kenton >>> >>> On Tue, Aug 20, 2019 at 7:53 AM Vitali Lovich <[email protected]> wrote: >>> >>>> Hi, I'm trying to use cap'n'proto as want efficient in-memory store of >>>> events that adds as little overhead as possible to the act of creating an >>>> event. Where I'm at is that I have per thread buffers that I in-place >>>> allocate a MallocMessageBuilder into a 512 buffer and use the remainder as >>>> scratch. I then defer serialization and destruction onto a background >>>> thread that flushes when there is a listener for the event (no listeners >>>> registered, simply overwrite). This works fine and performs fairly well >>>> (~300ns overhead per event on PC, maybe 1000ns on Android). >>>> >>>> Is there a way I can write a builder to live on the stack and not >>>> allocate it in the ring buffer in this setup? The challenges I ran into are >>>> that the builder zeroes on destruction and mandates that it's initialized >>>> with zeroed memory. >>>> >>>> I'm ok if this adds a restriction on the available arena size which I >>>> don't actually have now (since the builder itself is *very* at 200+ bytes >>>> relative to the events). I would still like to be able to add more data to >>>> the message in the background thread like I do now (eg process name, pid, >>>> etc) that is event-agnostic and can be cheaper to defer filling that in to >>>> the background listener delivery thread. >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Cap'n Proto" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/capnproto/CAF8PYMg54h9js%2BRiVhx4TA4DKiLty8iLZiRCUc21iKbmq7fhDQ%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/capnproto/CAF8PYMg54h9js%2BRiVhx4TA4DKiLty8iLZiRCUc21iKbmq7fhDQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- > You received this message because you are subscribed to the Google Groups > "Cap'n Proto" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/capnproto/CAF8PYMj435gc8xEgKS%3D%2BD%2BOiAFwTPfejiLHTM%3Dv%3D7cZkGYTkQA%40mail.gmail.com > <https://groups.google.com/d/msgid/capnproto/CAF8PYMj435gc8xEgKS%3D%2BD%2BOiAFwTPfejiLHTM%3Dv%3D7cZkGYTkQA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQk%2BgkqAZ_zib%2BYzbiX6QhZCEsSNUUJjsCeGswFWd0OuVQ%40mail.gmail.com.
