Thanks for the answer. I investigated the topic and looked into the codebase. As described in the docs, I found the mmap reader. What's still left though is an architectural question I'm not sure how to deal with appropriately. I have two applications communicating via CapnProto RPC and want to pass the mmap memory by this connection.
(1) Does it make more sense to just hand over the file identifier and read the message over there again or would it be more reasonable to transfer the memory via RPC but glue everything together with something like an mmap Client/Server. (2) If I need to make memory estimations beforehand for mmap, I haven't found a routine yet which provides functionality. Did I simply miss it or is there no message memory estimator. If not maybe this is worth a PR then. - Codie Am Donnerstag, 16. August 2018 20:17:45 UTC+2 schrieb Kenton Varda: > > Hi, > > Yes, Cap'n Proto should handle this use case much better than Protobufs. > By default there is a message size limit of 64MB, but this is for security > purposes, not performance, and you can easily configure a much larger limit > as needed using `ReaderOptions`. > > You could write your messages to a socket, but for the IPC use case you > can get even better performance using shared memory. If you set up a shared > memory region and build your Cap'n Proto message directly in it on the > sending end, then the receiving end can read it directly without ever > copying the bytes at all. With a socket, the bytes have to be copied at > least twice -- into and out of the kernel. > > -Kenton > > On Thu, Aug 16, 2018 at 1:52 AM, codie <[email protected] <javascript:> > > wrote: > >> Hello, >> I'm considering currently to use CapnProto for inter-process >> communication and could use a little hint before investing too much time >> into it. I have to send large vectors and matrices (~300MB) to another >> process. I know protobuf is not designed to handle such amounts of data for >> transfer. How about CapnProto? I also considered using simply e.g. >> sockets.Thanks. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Cap'n Proto" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> Visit this group at https://groups.google.com/group/capnproto. >> > > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. Visit this group at https://groups.google.com/group/capnproto.
