Unfortunately I'll be on a long flight, and cannot make it to the SIGBuild meeting. I'm definitely interested in the meeting notes and any follow-up meeting.
> I think we should leave CUDA out of the discussion initially and see if we can get the cpu-only wheel working correctly. Hopefully cpu-only is viable on manylinux2014 then we can tackle CUDA afterwards. 50% of the complexity is in the CUDA packaging. The other 50% is in shipping a more modern libstdc++.so I believe we'll make progress if we ignore CUDA, but we'll not address half of the issue. -- S On Mon, Feb 4, 2019 at 12:21 PM Antoine Pitrou <anto...@python.org> wrote: > > Le 04/02/2019 à 17:36, Uwe L. Korn a écrit : > > I think that problem is whether this would get merged. Conda was created > > after a meeting with Guido van Rossum and other folks at a PyCon quite > > some years ago where the final call was that this is not a problem of > > the core Python ecosystem and that the scientific Python community has > > to roll their own solution. > > > > @Wes McKinney <mailto:wesmck...@gmail.com> or someone else: Were you at > > this meeting and can outline why it was declined back then? > > I'm not sure anyone in this CC list was at that meeting (I wasn't). If > it's important to have the precise answer, I can try to CC someone. > > But I think the general answer is that it's a complex and difficult > endeavour, and the contribution structures inside the Python packaging > ecosystem, where most people are volunteers, didn't allow for it. > There's already enough lag maintaining the current software stack (pip > et al.). > > Anaconda then came up and became history, so to speak. > > Regards > > Antoine. > > > > > > Uwe > > > > Am Mo., 4. Feb. 2019 um 17:33 Uhr schrieb Manuel Klimek > > <kli...@google.com <mailto:kli...@google.com>>: > > > > On Mon, Feb 4, 2019 at 5:32 PM Uwe L. Korn <xho...@gmail.com > > <mailto:xho...@gmail.com>> wrote: > > > > Just as a heads-up: I would like to also join the meeting but am > > also located in Europe. > > > > I have spent quite some time with the packaging of wheels for > > pyarrow and turbodbc thus I would like to also give input on > > this. For Apache Arrow, I see newer manylinux2014 standard as a > > possible way to go. I'm not so fond of rolloing lib(std)c++ > > packages inside of pip. It's sadly the case that the features of > > pip don't allow a good dependency resolution, also with taking > > CUDA into account, a dependency resolution that differs between > > source and binary builds of a package. For this case, exactly > > conda was developed because it was considered out-of-scope for > > the core Python packaging system. I'm not sure whether we > > actually can fit all the requirements of the packages that take > > part in this mail thread into pip without simply reimplementing > > conda inside of pip. > > > > > > One question is probably: what would that entail, and why would it > > be bad? :) > > > > > > > > Uwe > > > > Am Mo., 4. Feb. 2019 um 16:34 Uhr schrieb Jason Zaman > > <ja...@perfinion.com <mailto:ja...@perfinion.com>>: > > > > yeah that's expected. The timing is complicated with people > > spread all > > over. We will post notes after the meeting on the SIG-Build > > mailing > > list and I'd also be up for organizing a separate call with > > europe > > folks if that would be of interest. > > > > On Mon, 4 Feb 2019 at 19:29, 'Manuel Klimek' via SIG Build > > <bu...@tensorflow.org <mailto:bu...@tensorflow.org>> wrote: > > > > > > +Dmitri Gribenko > > > > > > Dmitri has experience with EasyBuild, which seems to be > > used by the HPC community to solve the bootstrap problem and > > could be used to build a toolchain image & pip package. > > > > > > Unfortunately we'll not be able to join the meeting as > > it's at midnight CEST - looking forward to the conclusions > > from the meeting! > > > > > > On Mon, Feb 4, 2019 at 6:00 AM Jason Zaman > > <ja...@perfinion.com <mailto:ja...@perfinion.com>> wrote: > > >> > > >> Hey all, > > >> > > >> We're having the TensorFlow SIG-Build meeting on 5th Feb > > 3pm PST > > >> > > ( > https://www.timeanddate.com/worldclock/fixedtime.html?iso=20190205T15&p1=224 > ). > > >> Agenda is linked from: > > >> > > > https://groups.google.com/a/tensorflow.org/forum/#!topic/build/akyPcGoBIy4 > > >> > > >> I'd like to invite everyone from this thread to join the > > call if at > > >> all possible. The agenda for this meeting will spend most > > of the time > > >> focusing on the manylinux issue and hopefully we can get > > together to > > >> flesh out a decent plan on how to tackle this. > > >> > > >> Thanks, > > >> Jason > > >> > > >> On Wed, 30 Jan 2019 at 23:34, 'Manuel Klimek' via SIG > Build > > >> <bu...@tensorflow.org <mailto:bu...@tensorflow.org>> > wrote: > > >> > > > >> > On Wed, Jan 30, 2019 at 4:21 PM Antoine Pitrou > > <anto...@python.org <mailto:anto...@python.org>> wrote: > > >> >> > > >> >> > > >> >> Le 30/01/2019 à 16:09, Manuel Klimek a écrit : > > >> >> > > > >> >> > On Wed, Jan 30, 2019 at 3:51 PM Antoine Pitrou > > <anto...@python.org <mailto:anto...@python.org> > > >> >> > <mailto:anto...@python.org > > <mailto:anto...@python.org>>> wrote: > > >> >> > > > >> >> > > > >> >> > Le 30/01/2019 à 15:35, Manuel Klimek a écrit : > > >> >> > > > > >> >> > > Am I reading you wrong, or are you > > actually proposing to > > >> >> > package another > > >> >> > > libstdc++ version as a Python wheel? > > >> >> > > > > >> >> > > > > >> >> > > That would be the idea. > > >> >> > > > > >> >> > > > > >> >> > > If so, are you going to claim that the > > given wheel is > > >> >> > > manylinux-compatible? > > >> >> > > > > >> >> > > > > >> >> > > That is my question :) Why wouldn't it be? > > (I'd link it against > > >> >> > > manylinux libc and other C-only system libs) > > >> >> > > > >> >> > The problem is when you are loading two modules > > that link against > > >> >> > different libstdc++ versions in the same > > process. Incidentally, it's > > >> >> > the problem which prompted this discussion. > > >> >> > > > >> >> > > > >> >> > Sure, I'm aware :) I think as long as the > > requirement that all libraries > > >> >> > that want to exchange runtime-ABI-compatible > > versions are compiled with > > >> >> > the same toolchain, we can provide a way to mangle > > the symbols > > >> >> > differently. > > >> >> > > >> >> Ah, I see... Indeed, mangling the symbols may work for > > this. > > >> >> > > >> >> That said, if you're looking to create a de facto > > standard, why can't it > > >> >> be proposed as a manylinux iteration? > > >> > > > >> > > > >> > I'd have thought because it doesn't change the system > > requirements, while manylinux seems to be all about system > > requirements. > > >> > The idea is that that toolchain would still work on any > > manylinux compatible machine. > > >> > > > >> > > > >> > > > >> >> > > >> >> > > >> >> Regards > > >> >> > > >> >> Antoine. > > >> > > > >> > -- > > >> > You received this message because you are subscribed to > > the Google Groups "SIG Build" group. > > >> > To unsubscribe from this group and stop receiving > > emails from it, send an email to > > build+unsubscr...@tensorflow.org > > <mailto:build%2bunsubscr...@tensorflow.org>. > > >> > Visit this group at > > https://groups.google.com/a/tensorflow.org/group/build/. > > > > > > -- > > > You received this message because you are subscribed to > > the Google Groups "SIG Build" group. > > > To unsubscribe from this group and stop receiving emails > > from it, send an email to build+unsubscr...@tensorflow.org > > <mailto:build%2bunsubscr...@tensorflow.org>. > > > Visit this group at > > https://groups.google.com/a/tensorflow.org/group/build/. > > >
