One more question =)
I need to copy the root from a FdStream to a vector
Do I need to copy it into a MallocMessageBuilder ?
while (buff.tryGetReadBuffer().size() != 0) {
capnp::InputStreamMessageReader message(buff, capnp::ReaderOptions()); //,
scratch);
Message::Reader message = message.getRoot<Message>();
capnp::MallocMessageBuilder messageBuilder;
messageBuilder.setRoot(chunk);
messages->at(index++) = messageBuilder.getRoot<Message>();
}
On Thursday, July 20, 2017 at 3:40:29 PM UTC-7, Farid Zakaria wrote:
>
> Is MMAP the only way to randomly seek to an offset in the file?
>
> I can't seem to find a way to do that with kj::FdInputStream ?
>
>
> I'm trying to create an index of the elements in the file.
>
> On Thursday, July 20, 2017 at 2:42:44 PM UTC-7, Kenton Varda wrote:
>>
>> Hi Farid,
>>
>> Try using mmap() (disclaimer: haven't tried compiling this):
>>
>> struct stat stats;
>> KJ_SYSCALL(fstat(fd, &stats));
>> size_t size = stats.st_size;
>> const void* data = mmap(nullptr, size, PROT_READ, MAP_PRIVATE, fd, 0);
>>
>> if (data == MAP_FAILED) {
>> KJ_FAIL_SYSCALL("mmap");
>> }
>>
>> KJ_DEFER(KJ_SYSCALL(munmap(data, size)) { break; });
>>
>> KJ_SYSCALL(madvise(data, size, MADV_SEQUENTIAL));
>>
>>
>> kj::ArrayPtr<capnp::word> words(
>> reinterpret_cast<const capnp::word*>(data),
>> size / sizeof(capnp::word));
>>
>> while (words.size() > 0) {
>> capnp::FlatArrayMessageReader message(words);
>> Message::Reader chunk = message.getRoot<Message>();
>> count++;
>> words = kj::arrayPtr(message.getEnd(), words.end());
>> }
>>
>>
>> (On Windows you'll need to use CreateFileMapping() and MapViewOfFile() or
>> whatever.)
>>
>> -Kenton
>>
>> On Thu, Jul 20, 2017 at 2:11 PM, Farid Zakaria <[email protected]>
>> wrote:
>>
>>> Looking for some guidance on possibly the fastest way to read multiple
>>> messages in a file that has multiple MessageRoots
>>>
>>> So far I have this written:
>>>
>>> capnp::MallocMessageBuilder messageBuilder;
>>> //What is a good size for our words? As long as its smaller?
>>> capnp::word scratch[1024];
>>> kj::ArrayPtr<capnp::word> scratchSpace(scratch);
>>> kj::FdInputStream stream(fd);
>>> kj::BufferedInputStreamWrapper buff(stream);
>>>
>>> unsigned long count = 0;
>>> while (buff.tryGetReadBuffer().size() != 0) {
>>> capnp::InputStreamMessageReader message(buff, capnp::ReaderOptions(),
>>> scratchSpace);
>>> Message::Reader chunk = message.getRoot<Message>();
>>> count++;
>>> }
>>>
>>>
>>> I tried the helper methods first, but they seem too slow I think without
>>> the BufferedInputStreamWrapper.
>>>
>>>
>>> I don't do much C++ so I appreciate any help :)
>>>
>>> Btw, Is there an IRC chat or something ?
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Cap'n Proto" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> Visit this group at https://groups.google.com/group/capnproto.
>>>
>>
>>
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
Visit this group at https://groups.google.com/group/capnproto.
