Hi,
First time user of Cap'n Proto and I'm struggling a little bit. Perhaps I'm
just trying to use it in a way that it was never designed to be used.
I'm using a combination of boost::asio, ZeroMQ (via azmq — a boost::asio -
ØMQ bridge) and Cap'n Proto.
I'm trying to implement something that does collect & publish.
My application queries the status of multiple serial devices (using their
own protocols) and once I have their replies, I wish to broadcast a Cap'n
Proto message over the network via ØMQ that contains the device statuses.
The queries to the serial devices are done using asio's
async_write/async_read and are executed concurrently across multiple
threads to improve performance (multiple threads are running the asio IO
service).
My Cap'n Proto schema to represent these broadcast messages looks something
like this:
struct Status {
amp @0 :Amp;
hdmi @1 :HDMI;
…
struct Amp {
zones @0 :List(Zone);
struct Zone {
id @0 :Text;
volume @1 :UInt8;
channel @2 :UInt8;
…
}
}
struct HDMI {
outputs @0 :List(Output);
struct Output {
…
}
}
…
}
i.e, there is an "overall" status structure, which contains per-device
structs, etc.
I would like to build the Status message piecemeal — as each devices'
results appear, it builds it's respective sub-struct and makes it available
to whatever assembles and broadcasts the final, complete message.
I've modelled my code after asio — my device query functions are async, and
take a result handler function that is called once the results are
available.
This can be used in combination with asio's use_future feature, where the
async functions can be made to instead return a future containing the
result.
Using futures allows me to "scatter-gather" using boost::when_all(future,
future…) — i.e., execute all the query functions in parallel and obtain a
future representing their result, then perform the finishing operation (the
broadcast) once all the results are in.
To give you an idea of what I mean, here is the above in abridged
pseudo-code:
boost::shared_future<capnp::Orphan<::Status::Amp>> amp_status(capnp::
MallocMessageBuilder& mb) {
// query device async., possibly in a different thread, and complete
the future once the results are available
…
}
boost::shared_future<capnp::Orphan<::Status::HDMI>> hdmi_status(capnp::
MallocMessageBuilder& mb) {
…
}
void broadcast_status() {
capnp::MallocMessageBuilder mb;
auto status = mb.initRoot<Status>();
auto amp_status_f = amp_status(mb);
auto hdmi_status_f = hdmi_status(mb);
auto all_f = boost::when_all(amp_status_f, hdmi_status_f);
all_f.then([&](decltype(all_f)) {
status.adoptAmp(amp_status_f.get());
status.adoptHdmi(hdmi_status_f.get());
// publish message
});
}
Obviously, there's a few problems with all that.
My understanding, from reading the docs and source, is that capnp
MessageBuilder's aren't thread safe (well, I haven't found a mention that
they are thread-safe, so I'll assume they aren't)
Hence, each async query function would need it's own MessageBuilder. But
that then introduces more problems.
I'd have to pass around the MessageBuilder so it doesn't get destroyed, yet
MB's aren't copy-able — probably would have to be wrapped in a shared_ptr
or the like.
And unless I want the caller to guess/know the root message type of the
MessageBuilder, I need to pass around the root type's Builder too (or do I
still use Orphans?)
I also believe that it's not possible to adopt orphans between
MessageBuilders.
Alternatively I could have my own "in-memory" representation of the
per-devices statuses, and only right before I perform the broadcast does it
get converted to capnp.
Problem with that is it adds unnecessary duplication — the capnp schema +
the in-memory representation, which I thought was partly what capnp was
designed to avoid.
It also breaks the idea of abstraction — the message sending code has to
have intricate knowledge of every device and its status message format, so
it can convert between the in-memory rep and capnp.
I've done a little bit of reading of the group archives, and the
discussions on self-contained messages and POCS support caught my eye
(https://groups.google.com/d/topic/capnproto/Reg0wInHBdY/discussion).
What's the status of this? Is it built-in, and I've just overlooked it?
Any advice would be greatly appreciated.
Regards,
Adam
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
Visit this group at https://groups.google.com/group/capnproto.
