Hi Kenton,
Thank you. This works.
I ended up with the following:
class MessageSegmentsConstBufferSequenceAdapter {
private:
typedef const kj::ArrayPtr<const capnp::word> segment_t;
public:
typedef typename asio::const_buffer value_type;
typedef boost::transform_iterator<std::function<const
value_type(segment_t)>, segment_t*> const_iterator;
public:
MessageSegmentsConstBufferSequenceAdapter(const kj::ArrayPtr<segment_t>
a) : array(a) {}
const_iterator begin() const {
return const_iterator(array.begin(), &segment_buffer);
}
const_iterator end() const {
return const_iterator(array.end(), &segment_buffer);
}
private:
const kj::ArrayPtr<segment_t> array;
static asio::const_buffer segment_buffer(segment_t segment) {
return boost::asio::buffer(segment.begin(), segment.size() *
sizeof(capnp::word));
}
};
which can then be used like:
auto raw_segments = message.getSegmentsForOutput();
socket.async_send(MessageSegmentsConstBufferSequenceAdapter(raw_segments),
[](const boost::system::error_code& ec, std::size_t bytes_transferred){…});
Sorry for the late reply.
This is a hobby project and life got in the way the past few weeks!
- Adam
On Sun, Feb 12, 2017 at 7:48 AM, Kenton Varda <[email protected]> wrote:
> Hi Adam,
>
> I suspect you could write a wrapper class around ArrayPtr<const
> ArrayPtr<const word>> that implements ConstBufferSequence while adding
> minimal overhead. The class would have one data member of type
> ArrayPtr<...> and would implement the necessary methods to make it a
> ConstBufferSequence. You might need to similarly wrap each segment, if
> asio::buffer() would otherwise create a copy.
>
> BTW, careful about `asio::buffer(a.begin(), a.size())` -- I think you need
> to multiply the size by `sizeof(capnp::word)`, since asio::buffer seems to
> be byte-based.
>
> -Kenton
>
> On Sat, Feb 11, 2017 at 4:03 AM, <[email protected]> wrote:
>
>> Hi,
>>
>> I'm trying to send Cap'n Proto messages over boost::asio streams,
>> specifically ZeroMQ sockets.
>>
>> I'm currently using capnp::messageToFlatArray(…) combined with
>> boost::asio::buffer(…) like so:
>>
>> auto data = capnp::messageToFlatArray(message);
>> socket.async_send(boost::asio::buffer(data.begin(), data.size()), [](
>> const boost::system::error_code& ec, std::size_t bytes_transferred){…});
>>
>> While this works, the call to messageToFlatArray creates a copy of the
>> message contents.
>>
>> I found the following Stack Overflow reply from Kenton that briefly
>> outlined the idea of using MessageBuilder::getSegmentsForOutput()
>> combined with ZeroMQ multipart messages to avoid the overhead of a copy:
>>
>> Send the message as a ZeroMQ multi-part message, with the parts being the
>>> message's segments. capnp::MessageBuilder::getSegmentsForOutput()
>>> returns an array of arrays pointing into the raw message segments.
>>> capnp::SegmentArrayMessageReader takes such an array of arrays as
>>> input. If you can send the array of arrays as a multipart message, then you
>>> can skip using capnp/serialize.h at all, since its only purpose is to
>>> combine the segments into a single message with a segment table. In your
>>> case, ZeroMQ would be in charge of remembering where each segment starts
>>> and ends.
>>>
>>
>> -- Kenton Varda - http://stackoverflow.com/a/32042234
>>
>> All of asio's stream write functions take a ConstBufferSequence that
>> allows the caller to pass multiple buffers to be written in a single call.
>> For the ZeroMQ-asio binding (zmqp), passing multiple buffers to a write
>> call sends a multipart message.
>>
>> The result of MessageBuilder::getSegmentsForOutput() isn't
>> directly convertible to a ConstBufferSequence as the outer ArrayPtr
>> doesn't provide ::value_type or ::const_iterator and the inner ArrayPtr's
>> aren't convertible to asio::const_buffers.
>>
>> I've come up with the following code to do the conversion:
>>
>> auto raw_segments = message.getSegmentsForOutput();
>>
>> std::vector<asio::const_buffer> segments(raw_segments.size());
>> std::transform(raw_segments.begin(), raw_segments.end(), segments.begin
>> (), [](const kj::ArrayPtr<const capnp::word> a) {
>> return asio::buffer(a.begin(), a.size());
>> });
>>
>> socket.async_send(segments, [](const boost::system::error_code& ec,
>> std::size_t
>> bytes_transferred){});
>>
>> While this works, I do ask:
>> Is there an easier (or more "correct") way to convert the output of
>> MessageBuilder::getSegmentsForOutput() to a ConstBufferSequence?
>> Is there a more efficient method? Is it possible to forgo the conversion
>> of the outer array to a vector and the inner arrays to buffers? A
>> ConstBufferSequence is basically a "array of arrays" anyway.
>>
>>
>> Regards,
>> Adam
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> Visit this group at https://groups.google.com/group/capnproto.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
Visit this group at https://groups.google.com/group/capnproto.
