On Jul 19 2018, at 3:50 pm, Wes McKinney <wesmck...@gmail.com> wrote: > > Thanks Kou! > @Krisztian, where do we stand with the conda recipes in apache/arrow > vs. the ones in conda-forge? Will the updates there still be manual? >
Yes, I'm afraid so. Conda forge has a tightly coupled infrastructure with conda-smithy and the ci services (including circleci for linux packages which we cannot depend on - AFAIK). We can figure out something to keep the recipes/feedstocks in sync, but not sure it's worth the effort. > > @kou, the conda-forge repositories in question are: > https://github.com/conda-forge/arrow-cpp-feedstock > https://github.com/conda-forge/parquet-cpp-feedstock > https://github.com/conda-forge/pyarrow-feedstock > > Updating them is a bit laborious because we have arrow-cpp -> > parquet-cpp -> pyarrow dependency. So the updates must be completed > serially > > We should maybe post the voted-on release binaries at > anaconda.org/apache eventually but that doesn't need to happen this > release cycle > > - Wes > On Thu, Jul 19, 2018 at 7:03 AM, Uwe L. Korn <uw...@xhochy.com> wrote: > > Hi, > > > > I've added you to the PyPI admins. For the GPG key, please add it to the > > KEYS file in https://dist.apache.org/repos/dist/release/arrow/KEYS and > > https://dist.apache.org/repos/dist/dev/arrow/KEYS (they should be kept in > > sync). There is a small how-to at the top of the file. Just realised that I > > also need to do this. > > Uwe > > On Thu, Jul 19, 2018, at 12:12 PM, Kouhei Sutou wrote: > > > Hi, > > > > > > I want to take packaging related tasks but I'm busy between > > > 23th and 26th... I'll do at least the Ruby bindings related > > > tasks. Because 0.10.0 is the first release of the Ruby > > > bindings. > > > > > > I hope that I can do other packaging related tasks. > > > I have a PyPI account: https://pypi.org/user/kou/ > > > Do I need other accounts to release packages? conda-forge? > > > > > > I have a GPG key: https://github.com/kou.gpg > > > > > > Thanks, > > > -- > > > kou > > > > > > In <cajpuwmbpfm7yeckhu5uwdkmjhknqwirvbfufupzwwwhwq3m...@mail.gmail.com> > > > "Re: Need a release manager for Arrow 0.10.0 [was Re: Packaging status > > > update]" on Wed, 18 Jul 2018 15:13:46 -0400, > > > Wes McKinney <wesmck...@gmail.com> wrote: > > > > > > > If we could cut the release on the 24th (Tuesday) or 25th (Wednesday) > > > > that would be ideal. We're approaching T+4 months since 0.9.0 so I > > > > think we need to call pencils down once the last patches go in on > > > > Monday, barring any crazy blockers. At this rate I'm going to be > > > > working part of the weekend on this as we haven't managed to burn down > > > > the backlog to near zero yet (see > > > > https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.10.0+Release) > > > > > > > > In principle the packaging system should work fine against a tagged > > > > release because it uses setuptools_scm, so from Step 2 in > > > > https://github.com/apache/arrow/tree/master/dev/tasks#usage you would > > > > write something like > > > > > > > > git checkout apache-arrow-0.10.0 > > > > python dev/tasks/crossbow.py submit ... > > > > > > > > We should probably add a script in > > > > https://github.com/apache/arrow/tree/master/dev/release that will > > > > fetch the artifacts, sign them, and upload to the dist system. The tag > > > > and source release artifact will have already been tagged, built, > > > > signed, and uploaded by dev/release/02-source.sh. We can make sure > > > > these details are all documented and reproducible in > > > > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide > > > > > > > > Thanks Krisztian, Kou, Phillip for laboring on the packaging; I'm > > > > hopeful that this marks the beginning of a period of peace and > > > > prosperity for Arrow release managers so we can make releases roughly > > > > every month going forward. > > > > > > > > - Wes > > > > On Wed, Jul 18, 2018 at 2:51 PM, Krisztián Szűcs > > > > <szucs.kriszt...@gmail.com> wrote: > > > > > I won't be available between 27th and 31st. > > > > > > > > > > On Jul 18 2018, at 7:53 pm, Phillip Cloud <cpcl...@gmail.com> wrote: > > > > > > > > > > > > I can certainly help with the release vote. I'm out of town from > > > > > > the 20th > > > > > > to the 23rd, so it would be after that. Is there a particular > > > > > > target date > > > > > > we have in mind? > > > > > > > > > > > > On Wed, Jul 18, 2018 at 12:24 PM Wes McKinney <wesmck...@gmail.com> > > > > > > wrote: > > > > > > > hi Li, > > > > > > > Thanks for volunteering -- we need a PMC to sign the source and > > > > > > > binary > > > > > > > artifacts and upload to the dist system for the release. > > > > > > > Realistically > > > > > > > we should probably break up the tasks related to the release (the > > > > > > > release vote, and follow up updates to PyPI, conda-forge, etc. ) > > > > > > > -- > > > > > > > there's quite a lot of work to do before, during, and after the > > > > > > > vote. > > > > > > > @PhillipC, would you want to assist with the release vote process > > > > > > > since you've been engaged w/ Krisztian on the packaging work? > > > > > > > Since > > > > > > > you guys work in the same physical location it might make > > > > > > > coordination > > > > > > > easier > > > > > > > > > > > > > > - Wes > > > > > > > On Wed, Jul 18, 2018 at 12:00 PM, Li Jin <ice.xell...@gmail.com> > > > > > > > wrote: > > > > > > > > I am currently on vacation but will be back on 22th and I am > > > > > > > > not PMC. If > > > > > > > > these are not issues , I can volunteer to be RM for the release. > > > > > > > > > > > > > > > > Li > > > > > > > > On Wed, Jul 18, 2018 at 4:09 PM Wes McKinney > > > > > > > > <wesmck...@gmail.com> > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > hi folks, > > > > > > > > > Any takers? It would be good for this knowledge to be spread > > > > > > > > > to more > > > > > > > > > people. > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > Wes > > > > > > > > > > > > > > > > > > On Thu, Jul 12, 2018 at 1:11 PM, Wes McKinney > > > > > > > > > <wesmck...@gmail.com> > > > > > > > wrote: > > > > > > > > > > Can a member of the Arrow PMC volunteer to be the release > > > > > > > > > > manager for > > > > > > > > > > 0.10.0? There is some prep work to be done to make sure > > > > > > > > > > that you can > > > > > > > > > > produce the binary packages as part of the release vote; if > > > > > > > > > > this is > > > > > > > > > > done at the last minute it will probably delay the release. > > > > > > > > > > > > > > > > > > > > Thank you, > > > > > > > > > > Wes > > > > > > > > > > > > > > > > > > > > On Wed, Jul 11, 2018 at 12:10 PM, Wes McKinney > > > > > > > > > > <wesmck...@gmail.com> > > > > > > > > > wrote: > > > > > > > > > > > Thanks Krisztian for the update and all your work on this. > > > > > > > > > > > > > > > > > > > > > > We need a PMC member to step up to be the release manager > > > > > > > > > > > for 0.10. > > > > > > > > > > > Ideally they should undertake one or more dry runs of > > > > > > > > > > > simulating the > > > > > > > > > > > 0.10 release so we aren't faced with any surprises when > > > > > > > > > > > we go to cut > > > > > > > > > > > the release on ~July 23. > > > > > > > > > > > > > > > > > > > > > > I moved the release management guide to Confluence: > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide > > > > > > > > > > > > > > > > > > > > > > so we should ensure that there aren't any snowflake-y / > > > > > > > > > > > undocumented > > > > > > > > > > > aspects to the release build workflow. > > > > > > > > > > > > > > > > > > > > > > - Wes > > > > > > > > > > > On Wed, Jul 11, 2018 at 11:54 AM, Krisztián Szűcs > > > > > > > > > > > <szucs.kriszt...@gmail.com> wrote: > > > > > > > > > > > > Hi All! > > > > > > > > > > > > > > > > > > > > > > > > Just want to send a quick summary about the packaging. > > > > > > > > > > > > What's working: > > > > > > > > > > > > python wheel builds > > > > > > > > > > > > > > > > > > > > > > > > conda package builds (the recent conda forge changes > > > > > > > > > > > > caused some > > > > > > > > > unexpected problems) > > > > > > > > > > > > > > > > > > > > > > > > linux package builds (enriched with ubuntu bionic) > > > > > > > > > > > > scheduled daily builds (triggered by travis) > > > > > > > > > > > > query build statuses from the CLI (see output > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://bit.ly/2ztnSqV ( > > > > > > > > > > > > > > > > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3...@getmailspring.com/0?redirect=https%3A%2F%2Fbit.ly%2F2ztnSqV&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D > > > > > > > > > )) > > > > > > > > > > > > > > > > > > > > > > > > download and gpg sign the binary artifacts > > > > > > > > > > > > What's partially working: > > > > > > > > > > > > binary artifact uploading (most of them are uploaded > > > > > > > > > > > > though) > > > > > > > > > > > > > > > > > > > > > > > > We have package name collisions due to uploading > > > > > > > > > > > > multiple tasks' > > > > > > > > > artifacts to a single github release > > > > > > > > > > > > (see for example: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/kszucs/crossbow/releases/tag/build-160 ( > > > > > > > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3...@getmailspring.com/1?redirect=https%3A%2F%2Fgithub.com%2Fkszucs%2Fcrossbow%2Freleases%2Ftag%2Fbuild-160&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D) > > > > > > > ). > > > > > > > > > We used to postfix > > > > > > > > > > > > these assets with the particular architecture, but we > > > > > > > > > > > > need a more > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > robust solution without renaming > > > > > > > > > > > > anything (either by creating tarballs or uploading to > > > > > > > > > > > > multiple > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > release > > > > > > > > > tags). > > > > > > > > > > > > > > > > > > > > > > > > All-in-all it's in quite good shape, I'll work on to > > > > > > > > > > > > refactor the > > > > > > > > > artifact uploading and finish the > > > > > > > > > > > > remaining jira tasks. IMHO we should start to test the > > > > > > > > > > > > whole release > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > procedure from next Monday > > > > > > > > > > > > and ship a couple of test releases before We reach July > > > > > > > > > > > > 20th. > > > > > > > > > > > > > > > > > > > > > > > > - Krisztian
