Hi,
Thanks for this! You've saved me much head-scratching -- it didn't occur to
me to check if the PackedMessageReader was borrowing or taking ownership of
the input stream.
I've create a custom reader class that holds both an ArrayInputStream and a
PackedMessageReader and this seems to be working perfectly.
Cheers,
Jonathan.
On Thursday, April 6, 2023 at 9:31:43 PM UTC+1 ken...@cloudflare.com wrote:
> Hi Jonathan,
>
> I think the problem is with this function:
>
> std::shared_ptr<capnp::MessageReader> uda_capnp_deserialise(const char*
> bytes, size_t size)
> {
> // ArrayPtr requires non-const ptr, but we are only using this to read
> from the bytes array
>
> kj::ArrayPtr<kj::byte>
> buffer(reinterpret_cast<kj::byte*>(const_cast<char*>(bytes)), size);
> kj::ArrayInputStream in(buffer);
>
> return std::make_shared<capnp::PackedMessageReader>(in);
> }
>
> Here, you are returning a PackedMessageReader that is wrapping an
> ArrayInputStream, but the ArrayInputStream is allocated on the stack, so is
> no longer valid once the function returns. The stream needs to remain valid
> until the PackedMessageReader is destroyed. Cap'n Proto can actually lazily
> read segments of the message from the stream; it doesn't read the entire
> thing in the constructor.
>
> -Kenton
>
> On Thu, Apr 6, 2023 at 3:26 PM Jonathan Hollocombe <jholl...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I am using Cap'n Proto to serialise a data tree which is then sent as a
>> blob to a client to be unpacked. I was using something similar to:
>>
>> kj::VectorOutputStream out;
>> capnp::writePackedMessage(out, builder);
>> auto arr = out.getArray();
>> char* buffer = (char*)malloc(arr.size());
>> memcpy(buffer, arr.begin(), arr.size());
>>
>> and
>>
>> kj::ArrayPtr<kj::byte>
>> buffer(reinterpret_cast<kj::byte*>(const_cast<char*>(bytes)),
>> size);
>> kj::ArrayInputStream in(buffer);
>> capnp::PackedMessageReader reader{in};
>>
>> This is working for small tree sizes but segfaults when I get to larger
>> number of elements.
>>
>> I've found a solution that works by switching to messageToFlatArray and
>> FlatArrayMessageReader but this dramatically increases the amount of data
>> that needs to be sent, especially for small trees.
>>
>> I've also found that writing the buffer to a file and then using
>> PackedFdMessageReader also works but i'd rather avoid creating files
>> unnessarily.
>>
>> Is there a way to use the PackedMessageReader with a byte buffer?
>>
>> Cheers,
>> Jonathan.
>>
>> p.s. my example code can be found at:
>> https://github.com/jholloc/capnp_test
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to capnproto+...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/capnproto/ae6cafe1-f6f8-4eca-bab2-9ea3e73c3c8fn%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/capnproto/ae6cafe1-f6f8-4eca-bab2-9ea3e73c3c8fn%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/capnproto/0c469ce7-aea8-43b8-a5c4-1001aa1b5b4fn%40googlegroups.com.
