Hi Asheesh, I've set up a GitHub issue to track progress: https://github.com/capnproto/pycapnp/issues/197
-Jacob On Tuesday, December 10, 2019 at 11:48:51 PM UTC-8, Jacob Alexander wrote: > > Hi Asheesh, > > The manylinux targets look pretty interesting, docker isn't definitely the > most straight-forward way of handling glibc versions. Publishing to PyPi > with GitHub Actions does sound quite useful as well. > Let's sync up around December 23rd to see where things are at. > > -Jacob > > On Friday, December 6, 2019 at 7:59:31 PM UTC-8, asheesh wrote: >> >> Hi Jacob and all, >> >> From what I've seen, there are some Docker-based build environments that >> are designed to overcome the glibc version binary compatibility issues you >> mentioned. >> >> I'm semi offline right now, so I can't find references, but the thing I'd >> search for is [manylinux1 docker build]. >> >> Docker isn't essential, but it serves as a convenient way to distribute >> enough of an operating system to do these builds. >> >> I'm enthusiastic about helping you figure this out, so I'll watch this >> email thread. Also, if you want to remote pair program on it, I'm game the >> week of December 23, or any week in January. >> >> There's also a newer manylinux2010 build target if you need it. >> >> If you want to avoid a manual build step for you, I'd also be happy to >> help you set up GitHub Actions, Travis CI, or some other automated system >> for publishing those wheels (binary packages) to PyPI. >> >> Cheers, >> >> Asheesh. >> >> >> On Sat, Dec 7, 2019, 06:31 Jacob Alexander <[email protected]> wrote: >> >>> Cool! >>> >>> Yeah, I agree with bumping the version. >>> >>> One of the things I want to figure out is how to properly push binary >>> linux pypi packages to pypi.org. Basically glibc compatibility for >>> Python is a nightmare so there's a need to use a glibc as old as possible >>> (but also not run into issues with distros actually requiring a minimum >>> version of glibc for some things; I've been having issues with glibc 2.12.2 >>> and some newer distros for example). I vaguely recall there are some >>> Fortunately macOS and Windows are easy in this regard. >>> >>> -Jacob >>> >>> On Friday, December 6, 2019 at 3:13:17 PM UTC-8, Kenton Varda wrote: >>>> >>>> Hi Jacob, >>>> >>>> Sorry for the delay in replying. >>>> >>>> This is awesome work! >>>> >>>> I'd love to let you take over the official pycapnp repository and merge >>>> this back in. I think it's totally fine if you've made breaking API >>>> changes >>>> in the name of cleaning things up. Depending on the extent of the changes >>>> it might make sense to bump the major version number before the next >>>> release. The Go implementation is on version 3 now so there's plenty of >>>> precedent for that. >>>> >>>> I've filed an issue on pycapnp to propose making you a maintainer, to >>>> give the current owners a chance to weigh in: >>>> https://github.com/capnproto/pycapnp/issues/194 >>>> >>>> -Kenton >>>> >>>> On Mon, Dec 2, 2019 at 3:40 PM <[email protected]> wrote: >>>> >>>>> I don't mind helping maintain pycapnp. It will make it easier for me >>>>> if I can pull over most/all of my current changes. >>>>> >>>>> Some things that I've done >>>>> >>>>> - Removed most of the references to deprecated capnproto functions >>>>> (there are a few left that are a bit tricky that have to do with the >>>>> .capnp >>>>> file parser imports) >>>>> - Added Windows support (there seems to be a bug related to the >>>>> capnproto timer that's erroring out the last few tests) >>>>> - asyncio support for Python (this enables TLS support for both >>>>> client and servers) >>>>> - Python 3.7+ support (earlier versions don't work well with >>>>> asyncio and will take a bunch of work to make work) >>>>> - Updated the minimum version to capnproto 0.7.0 >>>>> - Moved from Travis to Github Actions >>>>> - No more git flow >>>>> - Updated to C++14 (fixes lots of issues, including those that >>>>> used to be there with macOS) >>>>> >>>>> This is my github https://github.com/haata >>>>> >>>>> I don't have many more changes planned. Well, beyond fixing new issues >>>>> that pop-up, adding support for new Python/OS options and the last few >>>>> Windows test errors. I also want to cleanup the documentation a bit and >>>>> add >>>>> a proper changelog. >>>>> >>>>> I'm also not sure how the package uploads work for >>>>> https://pypi.org/project/pycapnp/ and user accounts there. I have my >>>>> forked version here: https://pypi.org/project/pycapnp-async/ >>>>> >>>>> On Monday, December 2, 2019 at 1:29:20 PM UTC-8, Kenton Varda wrote: >>>>>> >>>>>> On Fri, Nov 29, 2019 at 1:28 AM Jacob Alexander <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> I'm happy to upstream (though I've taken some more extreme decisions >>>>>>> in some areas in regards to prior compatibility). >>>>>>> Unfortunately, I don't believe there's an active maintainer for the >>>>>>> pycapnp github repo so it's currently a bit futile. :( >>>>>>> >>>>>> >>>>>> A few months ago, Colin Jermain (https://github.com/cjermain; I >>>>>> don't seem to have his e-mail) offered to help with pycapnp maintenance, >>>>>> so >>>>>> I gave him commit rights. It looks like he did some work but hasn't >>>>>> touched >>>>>> it in a few months, so I'm not sure if he intends to keep working on it. >>>>>> If >>>>>> someone else wants to take over maintainership I'm happy to help get the >>>>>> right permissions set up. >>>>>> >>>>>> -Kenton >>>>>> >>>>>> >>>>>>> >>>>>>> On Friday, November 29, 2019 at 12:48:46 AM UTC-8, pepijn de vos >>>>>>> wrote: >>>>>>>> >>>>>>>> Thanks for the link. >>>>>>>> While I'm not actually interested in RPC or async at all, this fork >>>>>>>> actually works. >>>>>>>> Any chance this will be upstreamed? >>>>>>>> >>>>>>>> Pepijn >>>>>>>> >>>>>>>> On Fri, Nov 29, 2019 at 7:27 AM Jacob Alexander <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> While I haven't gotten all the tests working yet (there are some >>>>>>>>> issues with some of the timer functions on Windows it seems), I spent >>>>>>>>> a >>>>>>>>> bunch of time getting asyncio working with pycapnp (it was the most >>>>>>>>> reasonable way to get native Python TLS support working). >>>>>>>>> I've fixed a lot of bugs, removed a lot of the deprecated C++ >>>>>>>>> functions (the warnings were hiding a lot of serious issues). >>>>>>>>> >>>>>>>>> https://github.com/haata/pycapnp-async >>>>>>>>> >>>>>>>>> I'll be using this for a cross-platform tool I've been working so >>>>>>>>> I'll at least be maintaining it for basic client functionality >>>>>>>>> (server >>>>>>>>> stuff works as well, minus a few Windows tests). My main use case is >>>>>>>>> a TLS >>>>>>>>> connection between a Rust server and Python clients (this is >>>>>>>>> currently >>>>>>>>> working using tokio-rustls). >>>>>>>>> >>>>>>>>> It does require Python 3.7 and higher (3.8 also works). I >>>>>>>>> discovered some bugs in asyncio that make porting difficult to 3.5 >>>>>>>>> and 3.6 >>>>>>>>> (though it's likely possible with a bunch of effort). >>>>>>>>> >>>>>>>>> -HaaTa >>>>>>>>> >>>>>>>>> On Thursday, November 28, 2019 at 6:24:08 AM UTC-8, Pepijn de Vos >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Actually, it appears that the library contains a pointer, whereas >>>>>>>>>> the Python lib tries to use a reference. >>>>>>>>>> I'm not sure where this difference came from, but it's clearly >>>>>>>>>> incorrect. >>>>>>>>>> >>>>>>>>>> (env) [apicula]$ nm -gD /usr/lib/libkj-async-0.7.0.so | grep >>>>>>>>>> TransformPromiseNodeBase >>>>>>>>>> 000000000002b1e0 T >>>>>>>>>> _ZN2kj1_24TransformPromiseNodeBase7onReadyEPNS0_5EventE >>>>>>>>>> (env) [apicula]$ c++filt >>>>>>>>>> _ZN2kj1_24TransformPromiseNodeBase7onReadyEPNS0_5EventE >>>>>>>>>> kj::_::TransformPromiseNodeBase::onReady(kj::_::Event*) >>>>>>>>>> (env) [apicula]$ c++filt >>>>>>>>>> _ZN2kj1_24TransformPromiseNodeBase7onReadyERNS0_5EventE >>>>>>>>>> kj::_::TransformPromiseNodeBase::onReady(kj::_::Event&) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thursday, 28 November 2019 15:17:40 UTC+1, Pepijn de Vos wrote: >>>>>>>>>>> >>>>>>>>>>> Hey all, >>>>>>>>>>> >>>>>>>>>>> I'm exploring serialization libraries, so I installed the Python >>>>>>>>>>> lib and got the following error >>>>>>>>>>> >>>>>>>>>>> ImportError: [...]capnp.cpython-38-x86_64-linux-gnu.so: >>>>>>>>>>> undefined symbol: >>>>>>>>>>> _ZN2kj1_24TransformPromiseNodeBase7onReadyERNS0_5EventE >>>>>>>>>>> >>>>>>>>>>> Which is definitely a thing, and has been for two years: >>>>>>>>>>> https://github.com/capnproto/capnproto/blame/master/c%2B%2B/src/kj/async-inl.h#L392 >>>>>>>>>>> >>>>>>>>>>> It also seems to be linked correctly >>>>>>>>>>> >>>>>>>>>>> $ ldd [...]capnp.cpython-38-x86_64-linux-gnu.so >>>>>>>>>>> linux-vdso.so.1 (0x00007ffe675cf000) >>>>>>>>>>> libcapnpc-0.7.0.so => /usr/lib/libcapnpc-0.7.0.so ( >>>>>>>>>>> 0x00007f7dc5173000) >>>>>>>>>>> libcapnp-rpc-0.7.0.so => /usr/lib/libcapnp-rpc-0.7.0.so ( >>>>>>>>>>> 0x00007f7dc5090000) >>>>>>>>>>> libcapnp-0.7.0.so => /usr/lib/libcapnp-0.7.0.so ( >>>>>>>>>>> 0x00007f7dc4ff4000) >>>>>>>>>>> libkj-async-0.7.0.so => /usr/lib/libkj-async-0.7.0.so ( >>>>>>>>>>> 0x00007f7dc4f60000) >>>>>>>>>>> libkj-0.7.0.so => /usr/lib/libkj-0.7.0.so ( >>>>>>>>>>> 0x00007f7dc4eda000) >>>>>>>>>>> libstdc++.so.6 => /usr/lib/libstdc++.so.6 ( >>>>>>>>>>> 0x00007f7dc4cf0000) >>>>>>>>>>> libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f7dc4cd4000) >>>>>>>>>>> libc.so.6 => /usr/lib/libc.so.6 (0x00007f7dc4b0d000) >>>>>>>>>>> libpthread.so.0 => /usr/lib/libpthread.so.0 ( >>>>>>>>>>> 0x00007f7dc4aeb000) >>>>>>>>>>> libm.so.6 => /usr/lib/libm.so.6 (0x00007f7dc49a5000) >>>>>>>>>>> /usr/lib64/ld-linux-x86-64.so.2 (0x00007f7dc540e000) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> So the system library version is 0.7, and the latest Python >>>>>>>>>>> library is 0.6.4, not sure if the versions are just mismatched >>>>>>>>>>> somehow. >>>>>>>>>>> I tried installing from git with the same result, is the Python >>>>>>>>>>> library just outdated? >>>>>>>>>>> Or maybe the Arch package is just broken, because it doesn't >>>>>>>>>>> contain the required symbol? >>>>>>>>>>> >>>>>>>>>>> Cheers, >>>>>>>>>>> Pepijn >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to a topic in >>>>>>>>> the Google Groups "Cap'n Proto" group. >>>>>>>>> To unsubscribe from this topic, visit >>>>>>>>> https://groups.google.com/d/topic/capnproto/1x8_5_RC9wU/unsubscribe >>>>>>>>> . >>>>>>>>> To unsubscribe from this group and all its topics, send an email >>>>>>>>> to [email protected]. >>>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/capnproto/f8decb16-f304-4d80-9e2c-7df4946df458%40googlegroups.com >>>>>>>>> >>>>>>>>> <https://groups.google.com/d/msgid/capnproto/f8decb16-f304-4d80-9e2c-7df4946df458%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Cap'n Proto" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/capnproto/812fa7a9-0bf3-4eb4-b483-e20f8ebb33c0%40googlegroups.com >>>>>>> >>>>>>> <https://groups.google.com/d/msgid/capnproto/812fa7a9-0bf3-4eb4-b483-e20f8ebb33c0%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Cap'n Proto" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/capnproto/49839775-0604-4653-bda9-cb2c5d5f9d41%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/capnproto/49839775-0604-4653-bda9-cb2c5d5f9d41%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Cap'n Proto" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/capnproto/8ef1d626-d327-49aa-a666-f6d56e795884%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/capnproto/8ef1d626-d327-49aa-a666-f6d56e795884%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/5abdf35c-3eaa-4d36-b13e-1453444740ea%40googlegroups.com.
