messageToFlatArray() involves an extra copy compared to getSegmentsForOutput().
However, you are putting the bytes into a protobuf anyway. Protobuf will make many copies. So I don't think you should worry too much about this one extra copy. If you really want to use getSegmentsForOutput(), then you need to use a `repeated bytes` field in protobuf and you need to add each segment to the repeated bytes. -Kenton On Sun, Oct 6, 2019 at 8:00 PM 张小 <[email protected]> wrote: > hi Kenton： > getSegmentsForOutput() performance better than messageToFlatArray > right? These two interfaces What scenarios are applied to? > > best wish > whutbd > > 在 2019年10月7日星期一 UTC+8上午3:41:23，Kenton Varda写道： >> >> Your code only handles the case of a single segment. I recommend using >> the methods in capnp/serialize.h and org.capnproto.Serialize. I do not >> recommend using getSegmentsForOutput() nor constructing a MessageReader >> directly from a segment array, as these are advanced functions that can >> more easily go wrong. >> >> That said, in your example case, only a single segment should be needed, >> so something else is wrong too. >> >> I don't see any other obvious problems with the code you provided. I >> suspect a bug exists in the code you didn't show. I recommend you try to >> verify that the bytes on the receiving end are actually exactly the bytes >> from the sending end. Try logging the byte values and the size of the >> segment at each end to make sure everything matches. >> >> I think the most likely problem is that somewhere your `char*` is being >> interpreted as a NUL-terminated string, and is therefore being truncated at >> the first zero-valued byte. Most likely, the very first byte in the segment >> is a zero, so probably your ByteBuffer on the Java end ends up being >> zero-length. This probably leads to the struct appearing to contain only >> default values, hence goodsId is 0. >> >> -Kenton >> >> On Sat, Oct 5, 2019 at 11:28 PM 张小 <[email protected]> wrote: >> >>> hi Kenton： >>> I need your help,This problem has been bothering me for many days. >>> >>> In c++ server ,code like this >>> capnp::MallocMessageBuilder message; >>> FullInfo::Builder fullInfo = message.initRoot<FullInfo>(); >>> fullInfo.setGoodsId(919731) >>> >>> kj::ArrayPtr<const kj::ArrayPtr<const capnp::word>> segments >>> = message.getSegmentsForOutput(); >>> kj::ArrayPtr<const capnp::word> segment = segments[0]; >>> kj::ArrayPtr<const char> chars = segment.asChars(); >>> const char* mem_buf = chars.begin(); >>> int32_t mem_buf_len = chars.size(); >>> >>> and c++ server send the mem_buf to java_client by rpc call(bytes field) >>> >>> *Client and server communicate through protobuf rpc* >>> >>> In java_client code like this:() >>> >>> com.google.protobuf.ByteString capn_object_bytes = >>> response.getCapnObject(); >>> ByteBuffer[] bytes_buffer = new ByteBuffer[1]; >>> bytes_buffer[0] = capn_object_bytes.asReadOnlyByteBuffer(); >>> MessageReader message = new MessageReader(bytes_buffer, >>> ReaderOptions.DEFAULT_READER_OPTIONS); >>> Display.FullInfo.Reader fullInfo = >>> message.getRoot(Display.FullInfo.factory); >>> System.out.println("goodsId:" + fullInfo.getGoodsId()); >>> >>> but in java_client ，print goodsId is 0, however in c++ server, I >>> setGoodsId(919731) ,What's wrong with my method of use? @Kenton >>> >>> the fullinfo.capnp code is: >>> @0xcb70f505c89d1634; >>> >>> using Java = import "./compiler/src/main/schema/capnp/java.capnp"; >>> $Java.package("org.capnproto.examples"); >>> $Java.outerClassname("Display"); >>> >>> struct FullInfo{ >>> goodsId @1 :Int64; >>> } >>> >>> >>> >>> >>> >>> 在 2019年10月1日星期二 UTC+8上午3:23:37，Kenton Varda写道： >>>> >>>> Are you transmitting one big buffer containing all the segments, or are >>>> you transmitting each segment separately? >>>> >>>> It looks like you're trying to do the latter, but in that case you >>>> cannot use Serialize.read() to read it. You need to use `new >>>> MessageReader(segments)`. >>>> >>>> -Kenton >>>> >>>> On Mon, Sep 30, 2019 at 5:52 AM 张小 <[email protected]> wrote: >>>> >>>>> First Time: >>>>> In C++ server,I Use capnp::DynamicStruct, code like this: >>>>> capnp::MallocMessageBuilder msg; >>>>> capnp::DynamicStruct::Builder fullinfo_builder >>>>> = msg.initRoot<capnp::DynamicStruct>(g_schema); >>>>> capnp::DynamicStruct::Builder fullInfo = >>>>> msg.initRoot<capnp::DynamicStruct>(g_schema); >>>>> >>>>> capnp::messageToFlatArray(msg); ---->convert it to char* send to >>>>> java_cliet >>>>> >>>>> Java Client ,code like this: >>>>> MessageReader message >>>>> = org.capnproto.Serialize.read(capn_object_bytes.asReadOnlyByteBuffer());, >>>>> Info.Reader adInfo = message.getRoot(Info.factory); >>>>> >>>>> Above ， My First Time do work , but The Second Time, I do like this >>>>> ,it do not work: >>>>> In C++ server, code like this: >>>>> >>>>> capnp::MallocMessageBuilder message; >>>>> Info::Builder info = message.initRoot<Info>(); >>>>> info.setId(123); >>>>> kj::ArrayPtr<const kj::ArrayPtr<const capnp::word>> segments = >>>>> message.getSegmentsForOutput(); >>>>> >>>>> then I convert the segments object to char* send to java_client by >>>>> proto rpc_call (bytes field) >>>>> >>>>> >>>>> Java Client ,code like this: >>>>> MessageReader message >>>>> = org.capnproto.Serialize.read(capn_object_bytes.asReadOnlyByteBuffer());, >>>>> Info.Reader adInfo = message.getRoot(Info.factory); >>>>> >>>>> >>>>> In Second Time, happen Error like this: >>>>> >>>>> Exception in thread "main" java.lang.IllegalArgumentException >>>>> at java.nio.Buffer.limit(Buffer.java:275) >>>>> at org.capnproto.Serialize.read(Serialize.java:140) >>>>> at org.capnproto.Serialize.read(Serialize.java:111) >>>>> >>>>> How Can I do , In Second Time ,thanks >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> 在 2019年9月30日星期一 UTC+8下午8:29:41，David Renshaw写道： >>>>>> >>>>>> On the Java side, you need to first read the bytes into a >>>>>> `MessageReader`. That's typically done via one of the `Serialize.read()` >>>>>> methods. >>>>>> >>>>>> The `AnyPointer.Reader()` is not intended for external use. Probably >>>>>> we should make it private. >>>>>> >>>>>> >>>>>> On Mon, Sep 30, 2019 at 6:25 AM 张小 <[email protected]> wrote: >>>>>> >>>>>>> C++ server send Capn Object to JavaClient like this >>>>>>> >>>>>>> capnp::MallocMessageBuilder message; >>>>>>> Info::Builder info = message.initRoot<Info>(); >>>>>>> info.setId(123); >>>>>>> kj::ArrayPtr<const kj::ArrayPtr<const capnp::word>> segments = >>>>>>> message.getSegmentsForOutput(); >>>>>>> >>>>>>> then I convert the segments object to char* send to java_client by >>>>>>> proto rpc_call (bytes field) >>>>>>> >>>>>>> >>>>>>> Then In java_client, I do like this to read the capn object >>>>>>> com.google.protobuf.ByteString capn_object_bytes = >>>>>>> response.getCapnObject() >>>>>>> SegmentReader segment = new >>>>>>> SegmentReader(capn_object_bytes.asReadOnlyByteBuffer(), null); >>>>>>> AnyPointer.Reader any = new AnyPointer.Reader(segment, 0, >>>>>>> 64*1024*1024); >>>>>>> Info.Reader info = any.getAs(Info.factory); >>>>>>> System.out.println("Id:" + info.getId()); >>>>>>> >>>>>>> >>>>>>> run java_cliet ,happend error like this : >>>>>>> Exception in thread "main" java.lang.NullPointerException >>>>>>> at org.capnproto.WireHelpers.readStructPointer(WireHelpers.java:918) >>>>>>> at >>>>>>> org.capnproto.StructFactory.fromPointerReaderRefDefault(StructFactory.java:34) >>>>>>> at >>>>>>> org.capnproto.StructFactory.fromPointerReader(StructFactory.java:41) >>>>>>> at >>>>>>> org.capnproto.StructFactory.fromPointerReader(StructFactory.java:24) >>>>>>> at org.capnproto.AnyPointer$Reader.getAs(AnyPointer.java:56) >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Cap'n Proto" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/capnproto/081ab20b-5eae-43d5-a4fc-8230ef39d4a5%40googlegroups.com >>>>>>> <https://groups.google.com/d/msgid/capnproto/081ab20b-5eae-43d5-a4fc-8230ef39d4a5%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Cap'n Proto" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/capnproto/816c3227-c2b2-4b72-8ead-e707759a257c%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/capnproto/816c3227-c2b2-4b72-8ead-e707759a257c%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Cap'n Proto" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/capnproto/1e177ff5-e49f-4d4a-b5f3-1c096604bb29%40googlegroups.com >>> <https://groups.google.com/d/msgid/capnproto/1e177ff5-e49f-4d4a-b5f3-1c096604bb29%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "Cap'n Proto" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/capnproto/222eed4a-8f6a-48e7-828e-09235b228253%40googlegroups.com > <https://groups.google.com/d/msgid/capnproto/222eed4a-8f6a-48e7-828e-09235b228253%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQkmz35p6qr2EC4TnT%3DADWN_mDoXkum3B%3D9SjbrTSD1y2Q%40mail.gmail.com.
