Thanks for the quick response! I've filed an issue, https://github.com/jparyani/pycapnp/issues/137. It's easy to reproduce.
Should I take what you're saying to mean that in the absence of bugs the reader should only ever count data that is actually, "physically", read against the limit? So if, say, I have a large array of structs and I read a field of one of those structs, it's only the size of that field that gets counted -- not the size of the struct or the size of the entire array? c On Tue, Feb 14, 2017 at 1:00 PM, Kenton Varda <[email protected]> wrote: > Hi plesner, > > I don't think you are hitting the 2^63 limit. To do that you would have to > read over your whole 10MB of data 1000 times per second for 232 years. > > My guess is that there is indeed a bug in the Python library. Could you > file a bug against pycapnp, and include a test case to show which code path > you are hitting? > > -Kenton > > On Tue, Feb 14, 2017 at 1:26 AM, <[email protected]> wrote: > >> Over the weekend I migrated the android app I'm working on from using >> protobufs to capnproto messages and it's been great -- I zip the data >> anyway so it's not much larger and the infinite speedup when loading really >> makes a difference for what I'm doing. It's a game changer. >> >> There is a problem though. I generate all messages myself, they're >> relatively large (>10MB) and they are long-lived: days, possibly weeks, as >> long as the app lives. Initially I kept tripping the traversal limit and >> even after increasing the limit to 2^63 everywhere some of my tests still >> reliably fail because of it. It only happens with python tests, the java >> tests are fine, so that may be a bug in the python library but even if it >> is -- 2^63 isn't infinity, it can be hit within finite time and from a >> reliability standpoint it worries me that I don't know how to estimate a >> lower bound on that time. It seems to not be very long. >> >> What's the intended way for me to access my data without having to worry >> about tripping the limit? Should I open a new reader whenever I need to >> access my data? Or is there a way to convince myself that there is a limit >> I can set that will never be hit? >> >> Any input welcome. >> >> >> c >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Cap'n Proto" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> Visit this group at https://groups.google.com/group/capnproto. >> > > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. Visit this group at https://groups.google.com/group/capnproto.
