Another comment, I would also agree probably using raw data as the type is more appropriate, with a metadata information to encode the endianess.
This would be similiar to http://docs.ros.org/en/melodic/api/sensor_msgs/html/msg/PointCloud2.html On Thursday, 15 August 2019 at 3:25:09 pm UTC+8 Philipp Wissmann wrote: > Hi Kenton > > Thanks a lot for this amazingly fast reply and the explanations. > > We might try the direct pointer approach and it's very useful to know this > possibility but for now I think having the data in a Data member seems to > be working. > > > Cheers! > Philipp > > On Tuesday, August 13, 2019 at 7:43:08 PM UTC+2, Kenton Varda wrote: > >> Hi Philipp, >> >> This is a bit of an unusual case, where I imagine you are working with >> bulk vector data forming a 3D mesh or some such, and in order to hand if >> off to the graphics card, you really need a direct pointer to the >> underlying data and you need it to be in a specific layout. >> >> Using Data is this case might make sense. >> >> Alternatively, here's one trick I thought of: >> >> Say your Vector3f is defined as: >> >> struct Vector3f { x @0 :Float32; y @1 :Float32; z @2 :Float32; } >> >> Note that Cap'n Proto pads every struct to an 8-byte boundary, so there's >> 4 bytes of padding at the end of this struct. If that doesn't work for you, >> then I think you have no choice but to use Data. But if the padding is OK, >> then here's a way you can get a direct pointer to the data: >> >> const kj::byte* getRawPointer(capnp::List<Vector3f>::Reader list) { >> if (list.size() == 0) { >> return nullptr; >> } else { >> capnp::AnyStruct::Reader any(list[0]); >> KJ_REQUIRE(any.getPointerSection().size() == 0); >> KJ_REQUIRE(any.getDataSection().size() == 16); >> return any.getDataSection().begin(); >> } >> } >> >> Here, you're getting a direct pointer to the "data section" of the first >> struct in the list. Structs in a struct list are always stored >> contiguously, so you can extend this out to the size of the list. You do >> have to verify that the structs have the expected size, since technically >> struct sizes are allowed to change to support schema evolution (in this >> case, you'll never be able to add fields to Vector3f, except maybe a `w @3 >> :Float32` which would use the remaining padding without changing the size). >> All structs in a struct list have the same size, so if the first struct >> looks good, then the whole list is good. >> >> Hope that helps. >> >> -Kenton >> >> On Tue, Aug 13, 2019 at 9:32 AM <[email protected]> wrote: >> > Hi >>> >>> We used Cap'n Proto to serialize data in a shared memory environment and >>> defined some message types as structs containining List, i.e. >>> >>> struct Message{ >>> points @0 List(Vector3f) >>> }; >>> >>> Where Vector3f is another struct containing either 3 floats or a >>> List(Float32). However, extracting the Vector3f's from the List is not fast >>> enough for our use cases as you basically need to use std::copy or >>> std::transform on the List. We instead now replaced the definition by >>> >>> struct Message{ >>> points @0 Data; >>> }; >>> >>> and just read and copy the array of bytes. Unfortunately, this basically >>> gets rid of the nice schema language of Cap'n Proto. So, what's the most >>> efficient way to define and read a contiguous array of the same data type? >>> >>> Best, >>> Philipp >>> >>> *This message is confidential and only for the use of its addressee. >>> Email communications are not secure and therefore we do not accept >>> responsibility for the confidentiality or unaltered contents of this >>> message.* >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Cap'n Proto" group. >>> >> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >> >> >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/capnproto/6a847434-c48b-4b18-9252-737562301464%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/capnproto/6a847434-c48b-4b18-9252-737562301464%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/fd4b0e14-4282-47f1-812d-af4da6ad4dabn%40googlegroups.com.
