I'm working with some stuff right now in which there is a need for mass structured data transfer between two processes running on the same machine. They already have a protocol, and didn't use shared memory out of security concerns.
While I'm not in a position to get them to completely rethink the whole protocol at this time, I can certainly make it a recommendation for the future. Using memfds with memory sealing (which I will have to read about in more depth) might well be a good solution in the futrue. Unfortunately, there is a need to have some compatibility with a Windows implementation, but they can just use Cap'n Proto based named pipes or TCP connections and still be somewhat better off than the TCP-based protocol they use now. Right now they also move pointers around to point at structures read in off the wire. Those structures contain no pointers of course, but still... How much of a state machine is Cap'n Protocol internally? Can another transport be slid in underneath it really easily? How about if that transport multiplexes between several different senders, each of which is using Cap'n Proto, but which then have their own frame structure around it in which those frames may not correspond neatly to protocol units in Cap'n Proto (i.e. one Cap'n Proto protocol unit might be split between two frames)? On Tue, Jun 13, 2017 at 12:07 PM, Kenton Varda <[email protected]> wrote: > Indeed, this was a major design goal! > > I haven't personally used it this way yet, though I've used it for mmap() > many times, which has similar properties. > > I would like to develop an RPC transport which uses shared memory. It > would be especially neat if the code could automatically upgrade to shared > memory whenever it detects that it's communicating over a unix socket. > > One interesting hurdle for comms is what happens if you don't trust the > sending process. If the memory is still shared, they can potentially modify > the data while you're consuming it. If you make sure to read each bit of > data no more than once, then in theory no attack is possible -- but I > haven't yet reviewed the Cap'n Proto library itself to check that the > pointer validation reads each bit once, which would need to be guaranteed > before an application could consider relying on this. > > Linux's memfds support "sealing" the memory after writing it so that the > consuming end can be assured that no further changes are occurring. > However, messages need to be pretty large before this becomes worthwhile -- > for typical-size messages, an upfront memcpy() will be much faster. > > -Kenton > > On Tue, Jun 13, 2017 at 11:26 AM, Omnifarious <[email protected]> > wrote: > >> Cap'n Proto seems like it would be ideal for a shared memory >> communications channel, right down to the relative pointers. Has anybody >> used it this way? What are the hurdles? >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Cap'n Proto" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> Visit this group at https://groups.google.com/group/capnproto. >> > > -- Please only send email to this account that you consider public enough to be published on a public web page. Eric Hopper -- http://www.omnifarious.org/~hopper/ -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. Visit this group at https://groups.google.com/group/capnproto.
