I've added https://issues.apache.org/jira/browse/ARROW-4216 ("Python CUDA API docs") to the 0.12.0 release. There's a PR ready and it doesn't look contentious.
Regards Antoine. Le 10/01/2019 à 00:40, Wes McKinney a écrit : > I'll connect with Krisztian tomorrow (Thursday) morning to make sure > KEYS are in order and to give him the day to do a run through of the > release process to make sure there are no problems. In the meantime > I'll try to resolve as many of the remaining issues as possible, then > move the leftovers to 0.13. So we can start a vote on Friday > > On Wed, Jan 9, 2019 at 7:44 AM Antoine Pitrou <anto...@python.org> wrote: >> >> >> No preference from me. >> >> Regards >> >> Antoine. >> >> >> Le 09/01/2019 à 14:34, Krisztián Szűcs a écrit : >>> Should We aim for Friday (2019-01-11) or Thursday (2019-01-10)? >>> >>> On Wed, Jan 9, 2019 at 11:43 AM Antoine Pitrou <anto...@python.org> wrote: >>> >>>> >>>> For the record, none of these issues seem release-critical to me, apart >>>> from the RAT issue and perhaps the Windows git-describe issue. Though >>>> perhaps the release manager is best placed to evaluate them. >>>> >>>> Regards >>>> >>>> Antoine. >>>> >>>> >>>> Le 08/01/2019 à 23:07, Wes McKinney a écrit : >>>>> Well, a mid-week release candidate isn't looking too likely. I've >>>>> spent most of today working on the Gandiva Windows build and haven't >>>>> been able to do much 0.12 backlog items yesterday or today so far. >>>>> >>>>> Krisztian -- is your GPG key in KEYS yet? If not, you cannot cut a >>>> release. >>>>> >>>>> Out of the remaining issues in the backlog >>>>> https://issues.apache.org/jira/projects/ARROW/versions/12343858 >>>>> >>>>> Python issues >>>>> >>>>> * ARROW-2038: small comment to be addressed, then merged. Someone >>>>> please create follow up JIRA about CI for S3 >>>>> * ARROW-2298: I just nixed this for 0.12. There is a patch up but it >>>>> needs more work >>>>> * ARROW-2659 and ARROW-2860 are the same issue, I think. We should >>>>> fix as it has impacted many users and been reported many times >>>>> * ARROW-3344: Recurring Plasma test failure on Ubuntu 14.04. I will >>>>> take a look and see if the fix is difficult or not >>>>> * ARROW-3428: Close to merge ready, I will review and make sure all is >>>> good >>>>> * ARROW-4138: Windows rough edge, I will look >>>>> * ARROW-3916: There is a patch here; can someone look? Given the >>>>> overlap between Arrow and Parquet users it would be good to fix this >>>>> * ARROW-4181: Failing large_memory test -- can someone look (and also >>>>> see if any other large_memory tests are failing)? >>>>> >>>>> Rust items: can be merged but need not block release >>>>> >>>>> * ARROW-4040 >>>>> * ARROW-4193 >>>>> >>>>> The other issues >>>>> >>>>> * ARROW-3578: The release manager will need to be careful to see what >>>>> happens with RAT in the build >>>>> * ARROW-4199: patch available >>>>> * ARROW-854: To be merged with experimental designation on green build >>>>> * ARROW-4197: Emscripten issues with C++. Would be good to merge fix >>>>> in time for release >>>>> >>>>> I would say we should close the backlog by end-of-week at latest and >>>>> move forward with the release >>>>> >>>>> Thanks >>>>> Wes >>>>> >>>>> On Mon, Jan 7, 2019 at 10:08 AM Krisztián Szűcs >>>>> <szucs.kriszt...@gmail.com> wrote: >>>>>> >>>>>> Cool. I'm going to help with the python issues right after fixing the >>>> spark >>>>>> integration tests. >>>>>> >>>>>> On Mon, Jan 7, 2019 at 4:43 PM Wes McKinney <wesmck...@gmail.com> >>>> wrote: >>>>>> >>>>>>> Great, thank you! >>>>>>> >>>>>>> There are 20 0.12 issues Open or In-Progress, I'm going to tackle a >>>>>>> couple more Python things today and tomorrow, but let's see where we >>>>>>> stand by Wednesday or so and decide when to cut the release >>>>>>> >>>>>>> On Mon, Jan 7, 2019 at 4:38 AM Krisztián Szűcs >>>>>>> <szucs.kriszt...@gmail.com> wrote: >>>>>>>> >>>>>>>> Hey! >>>>>>>> >>>>>>>> On Fri, Jan 4, 2019 at 10:31 PM Wes McKinney <wesmck...@gmail.com> >>>>>>> wrote: >>>>>>>> >>>>>>>>> hi all, >>>>>>>>> >>>>>>>>> We should try to cut a release candidate for 0.12 as soon as >>>>>>>>> practical. Since we're just coming off the holidays, it would be good >>>>>>>>> to work for a few more business days to close out as many outstanding >>>>>>>>> patches as possible, and be in position to start a vote sometime next >>>>>>>>> week. >>>>>>>>> >>>>>>>>> There's a bunch of Python bugs in the backlog still -- if anyone can >>>>>>>>> pick up one or two of these it would be a help >>>>>>>>> >>>>>>>>> Would someone (Krisztian or Antoine maybe?) like to be the release >>>>>>> manager? >>>>>>>>> >>>>>>>> Yes, I volunteer :) >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> >>>>>>>>> On Wed, Dec 12, 2018 at 8:15 AM Wes McKinney <wesmck...@gmail.com> >>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> I agree that we should aim for time-based releases. Let's discuss a >>>>>>>>>> time-based release schedule (my preference would be ~every 2 months) >>>>>>>>>> for 2019 after we get 0.12 out. >>>>>>>>>> On Wed, Dec 12, 2018 at 3:15 AM Antoine Pitrou <anto...@python.org> >>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I think we should aim for time-based releases in general (rather >>>>>>> than a >>>>>>>>>>> specific set of features), but delaying this one sounds good to me. >>>>>>>>>>> >>>>>>>>>>> Regards >>>>>>>>>>> >>>>>>>>>>> Antoine. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Le 12/12/2018 à 01:34, Wes McKinney a écrit : >>>>>>>>>>>> hi all, >>>>>>>>>>>> >>>>>>>>>>>> I'm looking at the 0.12 backlog and I am not too comfortable >>>>>>> with the >>>>>>>>>>>> things that would have to be cut to get a release out next week. >>>>>>>>>>>> Additionally, not a lot of developers are going to be working the >>>>>>>>> week >>>>>>>>>>>> of December 24 because of the Christmas and New Year's holidays, >>>>>>> so >>>>>>>>>>>> even if we did release, it might not get seen by a lot of people >>>>>>>>> until >>>>>>>>>>>> after the New Year. >>>>>>>>>>>> >>>>>>>>>>>> Based on this, I would suggest we push to complete as much work >>>>>>> as >>>>>>>>>>>> possible (from the 0.12 backlog and beyond) by the end of the >>>>>>> year, >>>>>>>>>>>> and release as soon as possible in 2019. Of course, anyone is >>>>>>> welcome >>>>>>>>>>>> to contribute work that is not found in the 0.12 milestone =) >>>>>>>>>>>> >>>>>>>>>>>> Any objections? >>>>>>>>>>>> >>>>>>>>>>>> Thanks >>>>>>>>>>>> Wes >>>>>>>>>>>> On Mon, Dec 10, 2018 at 8:04 AM Andy Grove < >>>>>>> andygrov...@gmail.com> >>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Cool. I will continue to add primitive operations but I am now >>>>>>>>> adding this >>>>>>>>>>>>> in a separate source file to keep it separate from the core >>>>>>> array >>>>>>>>> code. >>>>>>>>>>>>> >>>>>>>>>>>>> I'm not sure how important it will be to support Rust data >>>>>>> sources >>>>>>>>> with >>>>>>>>>>>>> Gandiva. I can see that each language should be able to >>>>>>> construct >>>>>>>>> the >>>>>>>>>>>>> logical query plan to submit to Gandiva and let Gandiva handle >>>>>>>>> execution. I >>>>>>>>>>>>> think the more interesting part is how do we support >>>>>>>>> language-specific >>>>>>>>>>>>> lambda functions as part of that logical query plan. Maybe it is >>>>>>>>> possible >>>>>>>>>>>>> to compile the lambda down to LLVM (I haven't started learning >>>>>>>>> about LLVM >>>>>>>>>>>>> in detail yet so this is wild speculation on my part). Another >>>>>>>>> option is >>>>>>>>>>>>> for Gandiva to support calling into shared libraries and that >>>>>>> maybe >>>>>>>>> is >>>>>>>>>>>>> simpler for languages that support building C-native shared >>>>>>>>> libraries (Rust >>>>>>>>>>>>> supports this with zero overhead). >>>>>>>>>>>>> >>>>>>>>>>>>> Andy. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Sun, Dec 9, 2018 at 11:42 AM Wes McKinney < >>>>>>> wesmck...@gmail.com> >>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> hi Andy, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I can see an argument for having some basic native function >>>>>>> kernel >>>>>>>>>>>>>> support in Rust. One of the things that Gandiva has begun is a >>>>>>>>>>>>>> Protobuf-based serialized representation representation of >>>>>>>>> projection >>>>>>>>>>>>>> and filter expressions. In the long run I would like to see a >>>>>>> more >>>>>>>>>>>>>> complete relational algebra / logical query plan that can be >>>>>>>>> submitted >>>>>>>>>>>>>> for execution. There's complexities, though, such as bridging >>>>>>>>>>>>>> iteration of data sources written in Rust, say, with a query >>>>>>> engine >>>>>>>>>>>>>> written in C++. You would need to provide some kind of a >>>>>>> callback >>>>>>>>>>>>>> mechanism for the query engine to request the next chunk of a >>>>>>>>> dataset >>>>>>>>>>>>>> to be materialized. >>>>>>>>>>>>>> >>>>>>>>>>>>>> It will be interested to see what contributors will be >>>>>>> motivated >>>>>>>>>>>>>> enough to build over the next few years. At the end of the day, >>>>>>>>> Apache >>>>>>>>>>>>>> projects are do-ocracies. >>>>>>>>>>>>>> >>>>>>>>>>>>>> - Wes >>>>>>>>>>>>>> On Fri, Dec 7, 2018 at 6:22 AM Andy Grove < >>>>>>> andygrov...@gmail.com> >>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I've added one PR to the list ( >>>>>>>>> https://github.com/apache/arrow/pull/3119 >>>>>>>>>>>>>> ) >>>>>>>>>>>>>>> to update the project to use Rust 2018 Edition. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm also considering removing one PR from the list and would >>>>>>> like >>>>>>>>> to get >>>>>>>>>>>>>>> opinions here. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I have a PR (https://github.com/apache/arrow/pull/3033) to >>>>>>> add >>>>>>>>> some >>>>>>>>>>>>>> basic >>>>>>>>>>>>>>> math and comparison operators to primitive arrays. These are >>>>>>> baby >>>>>>>>> steps >>>>>>>>>>>>>>> towards implementing more query execution capabilities such as >>>>>>>>>>>>>> projection, >>>>>>>>>>>>>>> selection, etc but Chao made a good point that other Rust >>>>>>>>> implementations >>>>>>>>>>>>>>> don't have these kind of capabilities and I am now wondering >>>>>>> if >>>>>>>>> this is a >>>>>>>>>>>>>>> distraction. We already have Gandiva and the new efforts in >>>>>>> Ursa >>>>>>>>> labs and >>>>>>>>>>>>>>> it would probably make more sense to look at having Rust >>>>>>> bindings >>>>>>>>> for the >>>>>>>>>>>>>>> query execution capabilities there rather than having a >>>>>>> competing >>>>>>>>> (and >>>>>>>>>>>>>> less >>>>>>>>>>>>>>> capable) implementation in Rust. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thoughts? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Andy. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Thu, Dec 6, 2018 at 8:42 PM paddy horan < >>>>>>>>> paddyho...@hotmail.com> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Other than Andy’s PR below I’m going to try and find time to >>>>>>>>> work on >>>>>>>>>>>>>>>> ARROW-3827, I’ll bump it 0.13 if I can’t find the time early >>>>>>>>> next week. >>>>>>>>>>>>>>>> There is nothing else in the 0.12 backlog for Rust. It >>>>>>> would be >>>>>>>>> nice >>>>>>>>>>>>>> to >>>>>>>>>>>>>>>> get the parquet merge in though. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Paddy >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ________________________________ >>>>>>>>>>>>>>>> From: Andy Grove <andygrov...@gmail.com> >>>>>>>>>>>>>>>> Sent: Thursday, December 6, 2018 10:20:48 AM >>>>>>>>>>>>>>>> To: dev@arrow.apache.org >>>>>>>>>>>>>>>> Subject: Re: Timeline for Arrow 0.12.0 release >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I have PRs pending for all the Rust issues that I want to get >>>>>>>>> into >>>>>>>>>>>>>> 0.12.0 >>>>>>>>>>>>>>>> and would appreciate some reviews so I can go ahead and >>>>>>> merge: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> https://github.com/apache/arrow/pull/3033 (covers >>>>>>> ARROW-3880 and >>>>>>>>>>>>>>>> ARROW-3881 >>>>>>>>>>>>>>>> - add math and comparison operations to primitive arrays) >>>>>>>>>>>>>>>> https://github.com/apache/arrow/pull/3096 (ARROW-3885 - Rust >>>>>>>>> release >>>>>>>>>>>>>>>> process) >>>>>>>>>>>>>>>> https://github.com/apache/arrow/pull/3111 (ARROW-3838 - CSV >>>>>>>>> Writer) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> With these in place I plan on writing a tutorial for reading >>>>>>> a >>>>>>>>> CSV >>>>>>>>>>>>>> file, >>>>>>>>>>>>>>>> performing some operations on primitive arrays and writing >>>>>>> the >>>>>>>>> output >>>>>>>>>>>>>> to a >>>>>>>>>>>>>>>> new CSV file. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I am deferring ARROW-3882 (casting for primitive arrays) to >>>>>>>>> 0.13.0 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Andy. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Tue, Dec 4, 2018 at 7:57 PM Andy Grove < >>>>>>> andygrov...@gmail.com >>>>>>>>>> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I'd love to tackle the three related issues for supporting >>>>>>>>> simple >>>>>>>>>>>>>>>>> math/comparison operations on primitive arrays and casting >>>>>>>>> primitive >>>>>>>>>>>>>>>> arrays >>>>>>>>>>>>>>>>> but since the change to use Rust specialization feature I'm >>>>>>> a >>>>>>>>> bit >>>>>>>>>>>>>> stuck >>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>> need some assistance applying the math operations to the >>>>>>> numeric >>>>>>>>>>>>>> types >>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>> not the boolean primitives. I have added a comment to >>>>>>>>>>>>>>>>> https://github.com/apache/arrow/pull/3033 ... if I can get >>>>>>> help >>>>>>>>>>>>>> solving >>>>>>>>>>>>>>>>> for this PR then I should be able to handle the others. I'll >>>>>>>>> also do >>>>>>>>>>>>>> some >>>>>>>>>>>>>>>>> research and try and figure this out myself. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Andy. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Tue, Dec 4, 2018 at 7:03 PM Wes McKinney < >>>>>>>>> wesmck...@gmail.com> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Andy, Paddy, or other Rust developers -- could you review >>>>>>> the 6 >>>>>>>>>>>>>> issues >>>>>>>>>>>>>>>>>> in TODO in the 0.12 backlog and either assign them or move >>>>>>>>> them to >>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>> next release if they aren't going to be completed this >>>>>>> week or >>>>>>>>> next? >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Fri, Nov 30, 2018 at 4:34 PM Wes McKinney < >>>>>>>>> wesmck...@gmail.com> >>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> hi folks, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Tomorrow is December 1. The last major Arrow release >>>>>>> (0.11.0) >>>>>>>>> took >>>>>>>>>>>>>>>>>>> place on October 8. Given how much work has happened in >>>>>>> the >>>>>>>>>>>>>> project in >>>>>>>>>>>>>>>>>>> the last ~2 months, I think it would be great to complete >>>>>>> the >>>>>>>>> next >>>>>>>>>>>>>>>>>>> major release before the end-of-year holidays set in. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I've been curating the JIRA backlog the last couple of >>>>>>> weeks, >>>>>>>>> and >>>>>>>>>>>>>> have >>>>>>>>>>>>>>>>>>> just created a 0.12.0 release wiki page to help us stay >>>>>>>>> organized >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>> >>>> https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.12.0+Release >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Given that there are only 3 full working weeks between >>>>>>> now and >>>>>>>>>>>>>>>>>>> Christmas, I think we should be in position to cut a >>>>>>> release >>>>>>>>> by >>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> end of the week of December 10, i.e. by Friday December >>>>>>> 14. >>>>>>>>> Not >>>>>>>>>>>>>> all of >>>>>>>>>>>>>>>>>>> the TODO issues have to be completed to make the release, >>>>>>> but >>>>>>>>> it >>>>>>>>>>>>>> would >>>>>>>>>>>>>>>>>>> be good to push to complete as much as possible. Please >>>>>>> help >>>>>>>>> by >>>>>>>>>>>>>>>>>>> reviewing the backlog, and if possible, assigning issues >>>>>>> to >>>>>>>>>>>>>> yourself >>>>>>>>>>>>>>>>>>> that you'd like to pursue in the next 2 weeks. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Let me know if this sounds reasonable, or any concerns. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>>>>> Wes >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>> >>>>>>> >>>> >>>
