Protocol buffers are much better with mutable data. On May 24, 2016 12:16 PM, <[email protected]> wrote:
> I know this post is fairly old, but I'm curious - are you aware of any > serialization protocols that are more focused on catering mutable data? > > > On Friday, August 8, 2014 at 3:32:37 PM UTC-5, Kenton Varda wrote: >> >> Hi Wes, >> >> Yes, you can argue that certain use cases might require a copy where with >> protobufs they could be designed differently and avoid that copy. >> >> In my experience, most real-world uses of protobufs construct the whole >> message just before sending. Whether or not this construction makes sense >> to call a "copy" depends on the use case, but in any case that's not the >> copy that Cap'n Proto is claiming to avoid. >> >> If you are trying to store some in-memory state that changes over time, >> and you want to occasionally dump that state to disk / network, and you >> would normally have been happy keeping the state in a protobuf object, >> then, yes, with Cap'n Proto you now probably need to keep the state in some >> other structure and do an extra copy every time you dump it. However, this >> copy will still be a lot faster than protobuf encoding, since it's just a >> lot of loads and stores with few branches. >> >> Actually, though, if you're careful, you can in fact store your state in >> a Cap'n Proto structure. You just have to understand how Cap'n Proto >> manages memory. If the changes you're making to your state don't ever >> involve removing whole objects from the message tree, then there's no >> problem. Or, if you're willing to occasionally do a sort of "garbage >> collection" pass by making a copy of the whole message into a new >> MessageBuilder, then that can also solve the problem -- and may in fact >> turn out to be a lot more efficient than Protobuf memory management. For >> instance, you could do this "GC" pass every time the space occupied by the >> message doubles. This would give you amortized performance that's likely >> better than relying on malloc(). But you have to understand what you're >> doing, which is why we don't usually recommend it. :) >> >> -Kenton >> >> -- >> Sandstorm.io is crowdfunding! http://igg.me/at/sandstorm >> >> >> On Thu, Aug 7, 2014 at 6:09 PM, Wes Peters <[email protected]> wrote: >> >>> I've just finished reading your article comparing features of Cap'n >>> Proto with protobufs, SBE, and flatbuffers. >>> >>> I was involved in a selection trial for a serialization protocol two >>> years ago, and we went with the wrong too for all the "right" reasons: the >>> boss had already made a decision that we should be using XML to exchange >>> data. As you probably know, XML serialization in C++ is generally an >>> abysmal mess; I managed to make the best of it by discovering the quite >>> good XSD compiler. Eventually we switched from XML to Boost encoding for >>> most uses, because the XML encoding is just way too slow, so now we have >>> all the lovely overhead of describing messages in XSD without even getting >>> the questionable benefits of XML. I wanted to use protobufs, but got >>> shouted down because it wasn't XML. >>> >>> So, this experience has left with with a quandry for the next project. >>> I get to pick this time, and I say screw XML. So I'm out looking for >>> serializers *again.* >>> >>> One thing that leaped out to me in your chart: you point out that the >>> "zero copy" serialization libraries don't allow you to use the >>> protocol-compiler generated objects as mutable state. Doesn't this mean >>> that they are in fact *not* zero-copy then? If you have to maintain >>> the state in another object, then create an object from that for >>> serialization, you have in fact copied the data, probably in a >>> constructor. So what we've actually managed to do is complicate the >>> copying process... or have I mis-read what you wrote? >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Cap'n Proto" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> Visit this group at http://groups.google.com/group/capnproto. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "Cap'n Proto" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > Visit this group at https://groups.google.com/group/capnproto. > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. Visit this group at https://groups.google.com/group/capnproto.
