On Wed, 2018-02-28 at 18:25 -0800, Kenton Varda wrote: > On Wed, Feb 28, 2018 at 1:08 PM, <[email protected]> wrote: > > Hm. I don't see how you could do this without a full-on O(n) > > compaction pass on transmission/write-out. (Though you could argue > > that writing out the message is O(n) in the first place, so you don't > > use anything.) Even with one, I'm not sure I know how to do it (and > > still allow the message to grow arbitrarily in several places). > > I don't understand. What I described shouldn't change write logistics. > > Think of it this way: We're saying that far pointers can instead be > represented as regular pointers that happen to have an out-of-bounds > offset. Previously, such pointers were simply invalid. Now, we > interpret them as pointing into the next segment.
The difference is that previously you could resize a segment without adjusting any pointers. (Now, pointers into the next segment will change their targets.) Conversely, if you allocated too big a buffer for a segment, you could just ignore the extra part. (Now, again, you have to write/transmit the padding, or pointers into the next segment will get invalid.) > > Oh, I didn't think of that. Yes, that probably makes relative pointers > > worth keeping. You still need to get the message data into the buffer, > > and that's still O(n) [though a much faster O(n) than a tree traversal] > > unless you use something like scatter/gather, which is not exactly a > > standard feature of any of the current capnp libraries (or popular > > operating systems for that matter). > > Eh? kj::OutputStream and kj::AsyncOutputStream both support gather- > writes and Cap'n Proto uses it when serializing to a stream. This > translates into a writev() syscall. > > Moreover, the C++ implementation of Cap'n Proto supports linking a > large external byte blob into a message without copying it. The blob > becomes a message segment. Oh. Sorry. Should've studied the C++ API more carefully. I last did any C++ before C++0x, so I was too lazy to actually work through the somewhat unfamilliar style of the code and instead inferred the interfaces from a skim of the docs and what the format itself permits. Well, you see how that turned out. (For some reason I was thinking about Linux-specific asynchronous scatter/gather I/O, which _is_ a pain to use.) Alex PS. I'm still unclear as to what obscure case you were referring to regarding section sizes, I'll try to understand and reply at a saner time tomorrow. -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. Visit this group at https://groups.google.com/group/capnproto.
