For the record, none of these issues seem release-critical to me, apart from the RAT issue and perhaps the Windows git-describe issue. Though perhaps the release manager is best placed to evaluate them.
Regards Antoine. Le 08/01/2019 à 23:07, Wes McKinney a écrit : > Well, a mid-week release candidate isn't looking too likely. I've > spent most of today working on the Gandiva Windows build and haven't > been able to do much 0.12 backlog items yesterday or today so far. > > Krisztian -- is your GPG key in KEYS yet? If not, you cannot cut a release. > > Out of the remaining issues in the backlog > https://issues.apache.org/jira/projects/ARROW/versions/12343858 > > Python issues > > * ARROW-2038: small comment to be addressed, then merged. Someone > please create follow up JIRA about CI for S3 > * ARROW-2298: I just nixed this for 0.12. There is a patch up but it > needs more work > * ARROW-2659 and ARROW-2860 are the same issue, I think. We should > fix as it has impacted many users and been reported many times > * ARROW-3344: Recurring Plasma test failure on Ubuntu 14.04. I will > take a look and see if the fix is difficult or not > * ARROW-3428: Close to merge ready, I will review and make sure all is good > * ARROW-4138: Windows rough edge, I will look > * ARROW-3916: There is a patch here; can someone look? Given the > overlap between Arrow and Parquet users it would be good to fix this > * ARROW-4181: Failing large_memory test -- can someone look (and also > see if any other large_memory tests are failing)? > > Rust items: can be merged but need not block release > > * ARROW-4040 > * ARROW-4193 > > The other issues > > * ARROW-3578: The release manager will need to be careful to see what > happens with RAT in the build > * ARROW-4199: patch available > * ARROW-854: To be merged with experimental designation on green build > * ARROW-4197: Emscripten issues with C++. Would be good to merge fix > in time for release > > I would say we should close the backlog by end-of-week at latest and > move forward with the release > > Thanks > Wes > > On Mon, Jan 7, 2019 at 10:08 AM Krisztián Szűcs > <szucs.kriszt...@gmail.com> wrote: >> >> Cool. I'm going to help with the python issues right after fixing the spark >> integration tests. >> >> On Mon, Jan 7, 2019 at 4:43 PM Wes McKinney <wesmck...@gmail.com> wrote: >> >>> Great, thank you! >>> >>> There are 20 0.12 issues Open or In-Progress, I'm going to tackle a >>> couple more Python things today and tomorrow, but let's see where we >>> stand by Wednesday or so and decide when to cut the release >>> >>> On Mon, Jan 7, 2019 at 4:38 AM Krisztián Szűcs >>> <szucs.kriszt...@gmail.com> wrote: >>>> >>>> Hey! >>>> >>>> On Fri, Jan 4, 2019 at 10:31 PM Wes McKinney <wesmck...@gmail.com> >>> wrote: >>>> >>>>> hi all, >>>>> >>>>> We should try to cut a release candidate for 0.12 as soon as >>>>> practical. Since we're just coming off the holidays, it would be good >>>>> to work for a few more business days to close out as many outstanding >>>>> patches as possible, and be in position to start a vote sometime next >>>>> week. >>>>> >>>>> There's a bunch of Python bugs in the backlog still -- if anyone can >>>>> pick up one or two of these it would be a help >>>>> >>>>> Would someone (Krisztian or Antoine maybe?) like to be the release >>> manager? >>>>> >>>> Yes, I volunteer :) >>>> >>>>> >>>>> Thanks >>>>> >>>>> On Wed, Dec 12, 2018 at 8:15 AM Wes McKinney <wesmck...@gmail.com> >>> wrote: >>>>>> >>>>>> I agree that we should aim for time-based releases. Let's discuss a >>>>>> time-based release schedule (my preference would be ~every 2 months) >>>>>> for 2019 after we get 0.12 out. >>>>>> On Wed, Dec 12, 2018 at 3:15 AM Antoine Pitrou <anto...@python.org> >>>>> wrote: >>>>>>> >>>>>>> >>>>>>> I think we should aim for time-based releases in general (rather >>> than a >>>>>>> specific set of features), but delaying this one sounds good to me. >>>>>>> >>>>>>> Regards >>>>>>> >>>>>>> Antoine. >>>>>>> >>>>>>> >>>>>>> Le 12/12/2018 à 01:34, Wes McKinney a écrit : >>>>>>>> hi all, >>>>>>>> >>>>>>>> I'm looking at the 0.12 backlog and I am not too comfortable >>> with the >>>>>>>> things that would have to be cut to get a release out next week. >>>>>>>> Additionally, not a lot of developers are going to be working the >>>>> week >>>>>>>> of December 24 because of the Christmas and New Year's holidays, >>> so >>>>>>>> even if we did release, it might not get seen by a lot of people >>>>> until >>>>>>>> after the New Year. >>>>>>>> >>>>>>>> Based on this, I would suggest we push to complete as much work >>> as >>>>>>>> possible (from the 0.12 backlog and beyond) by the end of the >>> year, >>>>>>>> and release as soon as possible in 2019. Of course, anyone is >>> welcome >>>>>>>> to contribute work that is not found in the 0.12 milestone =) >>>>>>>> >>>>>>>> Any objections? >>>>>>>> >>>>>>>> Thanks >>>>>>>> Wes >>>>>>>> On Mon, Dec 10, 2018 at 8:04 AM Andy Grove < >>> andygrov...@gmail.com> >>>>> wrote: >>>>>>>>> >>>>>>>>> Cool. I will continue to add primitive operations but I am now >>>>> adding this >>>>>>>>> in a separate source file to keep it separate from the core >>> array >>>>> code. >>>>>>>>> >>>>>>>>> I'm not sure how important it will be to support Rust data >>> sources >>>>> with >>>>>>>>> Gandiva. I can see that each language should be able to >>> construct >>>>> the >>>>>>>>> logical query plan to submit to Gandiva and let Gandiva handle >>>>> execution. I >>>>>>>>> think the more interesting part is how do we support >>>>> language-specific >>>>>>>>> lambda functions as part of that logical query plan. Maybe it is >>>>> possible >>>>>>>>> to compile the lambda down to LLVM (I haven't started learning >>>>> about LLVM >>>>>>>>> in detail yet so this is wild speculation on my part). Another >>>>> option is >>>>>>>>> for Gandiva to support calling into shared libraries and that >>> maybe >>>>> is >>>>>>>>> simpler for languages that support building C-native shared >>>>> libraries (Rust >>>>>>>>> supports this with zero overhead). >>>>>>>>> >>>>>>>>> Andy. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Dec 9, 2018 at 11:42 AM Wes McKinney < >>> wesmck...@gmail.com> >>>>> wrote: >>>>>>>>> >>>>>>>>>> hi Andy, >>>>>>>>>> >>>>>>>>>> I can see an argument for having some basic native function >>> kernel >>>>>>>>>> support in Rust. One of the things that Gandiva has begun is a >>>>>>>>>> Protobuf-based serialized representation representation of >>>>> projection >>>>>>>>>> and filter expressions. In the long run I would like to see a >>> more >>>>>>>>>> complete relational algebra / logical query plan that can be >>>>> submitted >>>>>>>>>> for execution. There's complexities, though, such as bridging >>>>>>>>>> iteration of data sources written in Rust, say, with a query >>> engine >>>>>>>>>> written in C++. You would need to provide some kind of a >>> callback >>>>>>>>>> mechanism for the query engine to request the next chunk of a >>>>> dataset >>>>>>>>>> to be materialized. >>>>>>>>>> >>>>>>>>>> It will be interested to see what contributors will be >>> motivated >>>>>>>>>> enough to build over the next few years. At the end of the day, >>>>> Apache >>>>>>>>>> projects are do-ocracies. >>>>>>>>>> >>>>>>>>>> - Wes >>>>>>>>>> On Fri, Dec 7, 2018 at 6:22 AM Andy Grove < >>> andygrov...@gmail.com> >>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> I've added one PR to the list ( >>>>> https://github.com/apache/arrow/pull/3119 >>>>>>>>>> ) >>>>>>>>>>> to update the project to use Rust 2018 Edition. >>>>>>>>>>> >>>>>>>>>>> I'm also considering removing one PR from the list and would >>> like >>>>> to get >>>>>>>>>>> opinions here. >>>>>>>>>>> >>>>>>>>>>> I have a PR (https://github.com/apache/arrow/pull/3033) to >>> add >>>>> some >>>>>>>>>> basic >>>>>>>>>>> math and comparison operators to primitive arrays. These are >>> baby >>>>> steps >>>>>>>>>>> towards implementing more query execution capabilities such as >>>>>>>>>> projection, >>>>>>>>>>> selection, etc but Chao made a good point that other Rust >>>>> implementations >>>>>>>>>>> don't have these kind of capabilities and I am now wondering >>> if >>>>> this is a >>>>>>>>>>> distraction. We already have Gandiva and the new efforts in >>> Ursa >>>>> labs and >>>>>>>>>>> it would probably make more sense to look at having Rust >>> bindings >>>>> for the >>>>>>>>>>> query execution capabilities there rather than having a >>> competing >>>>> (and >>>>>>>>>> less >>>>>>>>>>> capable) implementation in Rust. >>>>>>>>>>> >>>>>>>>>>> Thoughts? >>>>>>>>>>> >>>>>>>>>>> Andy. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Thu, Dec 6, 2018 at 8:42 PM paddy horan < >>>>> paddyho...@hotmail.com> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Other than Andy’s PR below I’m going to try and find time to >>>>> work on >>>>>>>>>>>> ARROW-3827, I’ll bump it 0.13 if I can’t find the time early >>>>> next week. >>>>>>>>>>>> There is nothing else in the 0.12 backlog for Rust. It >>> would be >>>>> nice >>>>>>>>>> to >>>>>>>>>>>> get the parquet merge in though. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Paddy >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> ________________________________ >>>>>>>>>>>> From: Andy Grove <andygrov...@gmail.com> >>>>>>>>>>>> Sent: Thursday, December 6, 2018 10:20:48 AM >>>>>>>>>>>> To: dev@arrow.apache.org >>>>>>>>>>>> Subject: Re: Timeline for Arrow 0.12.0 release >>>>>>>>>>>> >>>>>>>>>>>> I have PRs pending for all the Rust issues that I want to get >>>>> into >>>>>>>>>> 0.12.0 >>>>>>>>>>>> and would appreciate some reviews so I can go ahead and >>> merge: >>>>>>>>>>>> >>>>>>>>>>>> https://github.com/apache/arrow/pull/3033 (covers >>> ARROW-3880 and >>>>>>>>>>>> ARROW-3881 >>>>>>>>>>>> - add math and comparison operations to primitive arrays) >>>>>>>>>>>> https://github.com/apache/arrow/pull/3096 (ARROW-3885 - Rust >>>>> release >>>>>>>>>>>> process) >>>>>>>>>>>> https://github.com/apache/arrow/pull/3111 (ARROW-3838 - CSV >>>>> Writer) >>>>>>>>>>>> >>>>>>>>>>>> With these in place I plan on writing a tutorial for reading >>> a >>>>> CSV >>>>>>>>>> file, >>>>>>>>>>>> performing some operations on primitive arrays and writing >>> the >>>>> output >>>>>>>>>> to a >>>>>>>>>>>> new CSV file. >>>>>>>>>>>> >>>>>>>>>>>> I am deferring ARROW-3882 (casting for primitive arrays) to >>>>> 0.13.0 >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Andy. >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Dec 4, 2018 at 7:57 PM Andy Grove < >>> andygrov...@gmail.com >>>>>> >>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I'd love to tackle the three related issues for supporting >>>>> simple >>>>>>>>>>>>> math/comparison operations on primitive arrays and casting >>>>> primitive >>>>>>>>>>>> arrays >>>>>>>>>>>>> but since the change to use Rust specialization feature I'm >>> a >>>>> bit >>>>>>>>>> stuck >>>>>>>>>>>> and >>>>>>>>>>>>> need some assistance applying the math operations to the >>> numeric >>>>>>>>>> types >>>>>>>>>>>> and >>>>>>>>>>>>> not the boolean primitives. I have added a comment to >>>>>>>>>>>>> https://github.com/apache/arrow/pull/3033 ... if I can get >>> help >>>>>>>>>> solving >>>>>>>>>>>>> for this PR then I should be able to handle the others. I'll >>>>> also do >>>>>>>>>> some >>>>>>>>>>>>> research and try and figure this out myself. >>>>>>>>>>>>> >>>>>>>>>>>>> Andy. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Dec 4, 2018 at 7:03 PM Wes McKinney < >>>>> wesmck...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Andy, Paddy, or other Rust developers -- could you review >>> the 6 >>>>>>>>>> issues >>>>>>>>>>>>>> in TODO in the 0.12 backlog and either assign them or move >>>>> them to >>>>>>>>>> the >>>>>>>>>>>>>> next release if they aren't going to be completed this >>> week or >>>>> next? >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Fri, Nov 30, 2018 at 4:34 PM Wes McKinney < >>>>> wesmck...@gmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> hi folks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Tomorrow is December 1. The last major Arrow release >>> (0.11.0) >>>>> took >>>>>>>>>>>>>>> place on October 8. Given how much work has happened in >>> the >>>>>>>>>> project in >>>>>>>>>>>>>>> the last ~2 months, I think it would be great to complete >>> the >>>>> next >>>>>>>>>>>>>>> major release before the end-of-year holidays set in. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I've been curating the JIRA backlog the last couple of >>> weeks, >>>>> and >>>>>>>>>> have >>>>>>>>>>>>>>> just created a 0.12.0 release wiki page to help us stay >>>>> organized >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>> https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.12.0+Release >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Given that there are only 3 full working weeks between >>> now and >>>>>>>>>>>>>>> Christmas, I think we should be in position to cut a >>> release >>>>> by >>>>>>>>>> the >>>>>>>>>>>>>>> end of the week of December 10, i.e. by Friday December >>> 14. >>>>> Not >>>>>>>>>> all of >>>>>>>>>>>>>>> the TODO issues have to be completed to make the release, >>> but >>>>> it >>>>>>>>>> would >>>>>>>>>>>>>>> be good to push to complete as much as possible. Please >>> help >>>>> by >>>>>>>>>>>>>>> reviewing the backlog, and if possible, assigning issues >>> to >>>>>>>>>> yourself >>>>>>>>>>>>>>> that you'd like to pursue in the next 2 weeks. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Let me know if this sounds reasonable, or any concerns. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>> Wes >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>> >>>
