Hi Kenton I prepared my source for sending to you. Can I send it to your email address? or reply to this group with it as attachment?
Thank you. 2021年2月21日日曜日 20:09:41 UTC+9 [email protected]: > Hi Kenton, > > Glad to see your reply. > > Yes I am aware of open mode. I do, for writing, > return _sopen_s(pfd, fpath.c_str(), (_O_BINARY | _O_CREAT | _O_RDWR | > _O_TRUNC), _SH_DENYNO, (_S_IREAD | _S_IWRITE)); > and for reading, > return _sopen_s(pfd, fpath.c_str(), (_O_BINARY | _O_RDWR), (_SH_DENYNO), > (_S_IREAD | _S_IWRITE)); > > Using _O_WRONLY for writing, _O_RDONLY for reading did not change the > situation. > For shared flag, I have tried SH_DENYNO or SH_DENYRW but made no > difference, as I had expected. > > I can provide my code. I will send it to you when its ready, have to clean > it a little I guess. > > Thank you very much for the message. > > 2021年2月20日土曜日 1:55:08 UTC+9 [email protected]: > >> Hi, >> >> Sorry for the slow reply. >> >> Just going to throw out a guess here, but seeing that you're using >> windows, did you make sure to use O_BINARY when opening the file >> descriptor? If you didn't, then the C library will silently corrupt your >> data by converting line endings. >> >> If that's not it, then could you provide a complete self-contained >> program (with main() function and all parameters filled in) that >> demonstrates the problem? >> >> -Kenton >> >> On Tue, Feb 16, 2021 at 7:15 AM [email protected] < >> [email protected]> wrote: >> >>> Hi >>> >>> I could do a little more things. >>> >>> Interestingly, capnp.exe built under x64 can do decode message. >>> I bet it does different processing than MessageReader's do and simply >>> follow message format structure. >>> >>> I made a test code using MapViewOfFile and FlatArrayMessageReader,. >>> It can read messages that I couldnt by using StreamFdMessageReader. >>> But easily fails with a little large data, say 32 entries in first level >>> List and 1 entry in the second level List, >>> getting following exception: >>> >>> Exception: capnp\serialize.c++:51: failed: expected array.size() >= >>> offset + segmentSize; Message ends prematurely in first segment. >>> stack: 7ff6a2af4b35 7ff6a2af3577 7ff6a2af14b6 7ff6a2b02323 7ffc1aa97c23 >>> 7ffc1b18d720 >>> >>> So I think I'm stuck. Any advice on how to proceed will be greatly >>> appreciated. >>> >>> Thank you in advance. >>> >>> >>> 2021年2月16日火曜日 16:23:38 UTC+9 [email protected]: >>> >>>> Hi >>>> >>>> So, trying to understand what is happening. >>>> >>>> using capnp.exe (32bit) in capnproto-tools-win32-0.8.0, message can be >>>> read correctly. >>>> capnp.exe decode sparse4d.capnp Dir4D < myoutput.bin >>>> By looking at hex dump of the message file, it looks like a proper >>>> message format. >>>> >>>> So, writing new message is okay but reading it is not working properly. >>>> Using capnproto-0.8.0 built with VS2017/x64 , and create a new message, >>>> save it to a file, and then reading it by my program fails. >>>> Since all happens when I do message.getRoot<Dir4D>(), this "Premature >>>> EOF" situation happens inside capnp library. >>>> >>>> Is there any way to fix this? or to narrow down what is wrong? >>>> >>>> Thank you. >>>> >>>> 2021年2月16日火曜日 4:56:44 UTC+9 [email protected]: >>>> >>>>> Hello >>>>> >>>>> I have been learning this wonderful software. >>>>> So I have been looking for an opportunity to use capnproto for my >>>>> project, >>>>> and now I have it but I need to use it with MSVC 2017(recently updated >>>>> to latest version 15.9.33), x64. >>>>> I only need serialization and no need for RPC or async. >>>>> This is my very first time posting a message to this community, please >>>>> let me know if I'm not doing right here. >>>>> >>>>> I have a schema whose basic construct is List of List of struct ( x,y >>>>> points in 4d expressed by two level lists). Second level will have a few >>>>> hundreds of entries and first level will have lots of entries. >>>>> And I am now stuck with a issue. If I put many entries in either of >>>>> lists (more than a few hundreds) and write that to a file, then when I >>>>> read >>>>> the message, I get exception like this: >>>>> Any help would be very appreciated. >>>>> >>>>> printSparseDir4D: std::exception: kj\io.c++:53: failed: expected n >= >>>>> minBytes; Premature EOF >>>>> stack: 7ff672faac46 7ff672fa3fd6 7ff672fa2de5 7ff672fa1308 >>>>> 7ff672fb15d3 7ffc1aa97c23 7ffc1b18d720 >>>>> >>>>> Now I found that writing 4 entries in first level, two entries in >>>>> second level (two x,y points each in 4 frames), I get the same error. >>>>> >>>>> I'm putting my schema and my read/write code below. I'm using these >>>>> codes with capnproto0.8.0. But actually this schema is a little striped >>>>> version than the one I want to use. When I used that a little more >>>>> complex >>>>> schema, I faced similar issue with capnproto 0.7.0 and 0.8.0. >>>>> >>>>> I built capnproto following the steps in the website, had to skip >>>>> heavy-tests project and kj-async (0.8.0), hoping the build was ok. >>>>> >>>>> Is there anything missing or done wrong? >>>>> It would be very much appreciated for any help. >>>>> Thank you. >>>>> >>>>> >>>>> sparse4d.capnp : >>>>> >>>>> # unique file ID, generated by `capnp id` >>>>> @0xae7dda29a38d355f; >>>>> >>>>> struct Sparse4DHeader { >>>>> tSize @0 :Int32; >>>>> zSize @1 :Int32; >>>>> ySize @2 :Int32; >>>>> xSize @3 :Int32; >>>>> } >>>>> >>>>> struct Dir4D { >>>>> header @0 :Sparse4DHeader; >>>>> planes @1 :List(SparsePlane); >>>>> >>>>> struct SparsePlane { >>>>> t @0 :UInt16; >>>>> z @1 :UInt16; >>>>> points @2 :List(Point); >>>>> } >>>>> } >>>>> >>>>> struct Point { >>>>> x @0 :UInt16; >>>>> y @1 :UInt16; >>>>> intensity @2 :Float32; >>>>> } >>>>> >>>>> >>>>> And read/write code: >>>>> >>>>> /** >>>>> * Copyright (c) 2021 ASTOM R&D >>>>> * Author: Yojiro Nobukuni >>>>> */ >>>>> //#include "pch.h" >>>>> >>>>> #include "../sparse4d.capnp.h" >>>>> #include <capnp/message.h> >>>>> #include <capnp/serialize-packed.h> >>>>> #include <iostream> >>>>> #include <ios> >>>>> >>>>> #include <sys/types.h> >>>>> #include <sys/stat.h> >>>>> #include <fcntl.h> >>>>> //#include <unistd.h> >>>>> >>>>> /** >>>>> * g++ writer.cpp photodeco_kernel.capnp.c++ -o writer -lcapnp -lkj >>>>> * ./writer > test.message >>>>> */ >>>>> >>>>> /** >>>>> * Write PhotodecoKernel message to fd >>>>> * @param int fd fd to read message in >>>>> */ >>>>> void writeSparse4D(int fd, unsigned int nofPlanes, unsigned int >>>>> nofPoints, bool packed) >>>>> { >>>>> ::capnp::MallocMessageBuilder message; >>>>> >>>>> auto dir4d = message.initRoot<Dir4D>(); >>>>> auto header = dir4d.initHeader(); >>>>> header.setTSize(7); >>>>> header.setZSize(7); >>>>> header.setXSize(7); >>>>> header.setYSize(7); >>>>> >>>>> auto planes = dir4d.initPlanes(nofPlanes); >>>>> for (auto plane : planes) { >>>>> plane.setT(9); >>>>> plane.setZ(8); >>>>> auto points = plane.initPoints(nofPoints); >>>>> #if 0 >>>>> // This one or the other one in #else block, same >>>>> behavior. >>>>> for (unsigned int i = 0; i < nofPoints; i++) { >>>>> points[i].setX(1); >>>>> points[i].setY(2); >>>>> points[i].setIntensity(3); >>>>> } >>>>> #else >>>>> for (auto p : points) { >>>>> p.setX(1); >>>>> p.setY(2); >>>>> p.setIntensity(3); >>>>> } >>>>> #endif >>>>> } >>>>> >>>>> try { >>>>> if (packed) >>>>> writePackedMessageToFd(fd, message); >>>>> else >>>>> writeMessageToFd(fd, message); >>>>> } >>>>> catch (std::exception& e) { >>>>> std::cout << "writeSparse4D: std::exception: " << e.what() << >>>>> std::endl; >>>>> return; >>>>> } >>>>> } >>>>> >>>>> >>>>> /** >>>>> * Print PhotodecoKernerl kernel in text >>>>> * @param PhotodecoKernel::Reader kernel >>>>> */ >>>>> void printDir4D(Dir4D::Reader dir4d, bool printPoints) >>>>> { >>>>> auto header = dir4d.getHeader(); >>>>> std::cout << " sizes[T,Z,Y,X] = " >>>>> << header.getTSize() << ", " << header.getZSize() << ", " << >>>>> header.getYSize() << ", " << header.getXSize() << ", " >>>>> << std::endl; >>>>> //std::cout << " nofPoints = " << header.getNofPoints() << std::endl; >>>>> >>>>> if (!printPoints) return; >>>>> >>>>> size_t nofPoints = 0; >>>>> auto planes = dir4d.getPlanes(); >>>>> for (auto plane : planes) { >>>>> const auto t = plane.getT(); >>>>> const auto z = plane.getZ(); >>>>> std::cout << "plane(t,z): " << t << ", " << z << std::endl; >>>>> for (auto p : plane.getPoints()) { >>>>> std::cout << t << ", " << z << ", " << p.getY() << ", " << p.getX() << >>>>> ", " << p.getIntensity() << std::endl; >>>>> nofPoints++; >>>>> } >>>>> } >>>>> >>>>> std::cout << "Total nof KernelPoints = " << nofPoints << std::endl; >>>>> } >>>>> >>>>> /** >>>>> * Read PhotodecoKernerl message from fd and print it in text >>>>> * @param int fd >>>>> * @param bool packed >>>>> */ >>>>> void printSparseDir4D(int fd, bool printPoints, bool packed) >>>>> { >>>>> std::cout << "printSparseDir4D" << std::endl; >>>>> >>>>> // readerOptions for reading message. >>>>> capnp::ReaderOptions readerOptions; >>>>> readerOptions.traversalLimitInWords = 1000 * 1000 * 1000 * >>>>> readerOptions.traversalLimitInWords; >>>>> //readerOptions.nestingLimit; >>>>> >>>>> ::capnp::MessageReader *message; >>>>> try { >>>>> if (packed) >>>>> message = new ::capnp::PackedFdMessageReader(fd, readerOptions); >>>>> else >>>>> message = new ::capnp::StreamFdMessageReader(fd, readerOptions); >>>>> >>>>> auto dir4d = message->getRoot<Dir4D>(); >>>>> printDir4D(dir4d, printPoints); >>>>> } >>>>> catch (std::exception& e) { >>>>> std::cout << "printSparseDir4D: std::exception: " << e.what() << >>>>> std::endl; >>>>> return; >>>>> } >>>>> >>>>> >>>>> delete message; >>>>> } >>>>> >>>>> >>>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Cap'n Proto" group. >>> >> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >> >> >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/capnproto/2d1ae382-e739-4ad0-9e74-cbdcc0ab1a0fn%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/capnproto/2d1ae382-e739-4ad0-9e74-cbdcc0ab1a0fn%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/854ef4ca-17b6-4776-bd4d-03340c75dbbcn%40googlegroups.com.
