No preference from me.
Regards Antoine. Le 09/01/2019 à 14:34, Krisztián Szűcs a écrit : > Should We aim for Friday (2019-01-11) or Thursday (2019-01-10)? > > On Wed, Jan 9, 2019 at 11:43 AM Antoine Pitrou <anto...@python.org> wrote: > >> >> For the record, none of these issues seem release-critical to me, apart >> from the RAT issue and perhaps the Windows git-describe issue. Though >> perhaps the release manager is best placed to evaluate them. >> >> Regards >> >> Antoine. >> >> >> Le 08/01/2019 à 23:07, Wes McKinney a écrit : >>> Well, a mid-week release candidate isn't looking too likely. I've >>> spent most of today working on the Gandiva Windows build and haven't >>> been able to do much 0.12 backlog items yesterday or today so far. >>> >>> Krisztian -- is your GPG key in KEYS yet? If not, you cannot cut a >> release. >>> >>> Out of the remaining issues in the backlog >>> https://issues.apache.org/jira/projects/ARROW/versions/12343858 >>> >>> Python issues >>> >>> * ARROW-2038: small comment to be addressed, then merged. Someone >>> please create follow up JIRA about CI for S3 >>> * ARROW-2298: I just nixed this for 0.12. There is a patch up but it >>> needs more work >>> * ARROW-2659 and ARROW-2860 are the same issue, I think. We should >>> fix as it has impacted many users and been reported many times >>> * ARROW-3344: Recurring Plasma test failure on Ubuntu 14.04. I will >>> take a look and see if the fix is difficult or not >>> * ARROW-3428: Close to merge ready, I will review and make sure all is >> good >>> * ARROW-4138: Windows rough edge, I will look >>> * ARROW-3916: There is a patch here; can someone look? Given the >>> overlap between Arrow and Parquet users it would be good to fix this >>> * ARROW-4181: Failing large_memory test -- can someone look (and also >>> see if any other large_memory tests are failing)? >>> >>> Rust items: can be merged but need not block release >>> >>> * ARROW-4040 >>> * ARROW-4193 >>> >>> The other issues >>> >>> * ARROW-3578: The release manager will need to be careful to see what >>> happens with RAT in the build >>> * ARROW-4199: patch available >>> * ARROW-854: To be merged with experimental designation on green build >>> * ARROW-4197: Emscripten issues with C++. Would be good to merge fix >>> in time for release >>> >>> I would say we should close the backlog by end-of-week at latest and >>> move forward with the release >>> >>> Thanks >>> Wes >>> >>> On Mon, Jan 7, 2019 at 10:08 AM Krisztián Szűcs >>> <szucs.kriszt...@gmail.com> wrote: >>>> >>>> Cool. I'm going to help with the python issues right after fixing the >> spark >>>> integration tests. >>>> >>>> On Mon, Jan 7, 2019 at 4:43 PM Wes McKinney <wesmck...@gmail.com> >> wrote: >>>> >>>>> Great, thank you! >>>>> >>>>> There are 20 0.12 issues Open or In-Progress, I'm going to tackle a >>>>> couple more Python things today and tomorrow, but let's see where we >>>>> stand by Wednesday or so and decide when to cut the release >>>>> >>>>> On Mon, Jan 7, 2019 at 4:38 AM Krisztián Szűcs >>>>> <szucs.kriszt...@gmail.com> wrote: >>>>>> >>>>>> Hey! >>>>>> >>>>>> On Fri, Jan 4, 2019 at 10:31 PM Wes McKinney <wesmck...@gmail.com> >>>>> wrote: >>>>>> >>>>>>> hi all, >>>>>>> >>>>>>> We should try to cut a release candidate for 0.12 as soon as >>>>>>> practical. Since we're just coming off the holidays, it would be good >>>>>>> to work for a few more business days to close out as many outstanding >>>>>>> patches as possible, and be in position to start a vote sometime next >>>>>>> week. >>>>>>> >>>>>>> There's a bunch of Python bugs in the backlog still -- if anyone can >>>>>>> pick up one or two of these it would be a help >>>>>>> >>>>>>> Would someone (Krisztian or Antoine maybe?) like to be the release >>>>> manager? >>>>>>> >>>>>> Yes, I volunteer :) >>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> On Wed, Dec 12, 2018 at 8:15 AM Wes McKinney <wesmck...@gmail.com> >>>>> wrote: >>>>>>>> >>>>>>>> I agree that we should aim for time-based releases. Let's discuss a >>>>>>>> time-based release schedule (my preference would be ~every 2 months) >>>>>>>> for 2019 after we get 0.12 out. >>>>>>>> On Wed, Dec 12, 2018 at 3:15 AM Antoine Pitrou <anto...@python.org> >>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> I think we should aim for time-based releases in general (rather >>>>> than a >>>>>>>>> specific set of features), but delaying this one sounds good to me. >>>>>>>>> >>>>>>>>> Regards >>>>>>>>> >>>>>>>>> Antoine. >>>>>>>>> >>>>>>>>> >>>>>>>>> Le 12/12/2018 à 01:34, Wes McKinney a écrit : >>>>>>>>>> hi all, >>>>>>>>>> >>>>>>>>>> I'm looking at the 0.12 backlog and I am not too comfortable >>>>> with the >>>>>>>>>> things that would have to be cut to get a release out next week. >>>>>>>>>> Additionally, not a lot of developers are going to be working the >>>>>>> week >>>>>>>>>> of December 24 because of the Christmas and New Year's holidays, >>>>> so >>>>>>>>>> even if we did release, it might not get seen by a lot of people >>>>>>> until >>>>>>>>>> after the New Year. >>>>>>>>>> >>>>>>>>>> Based on this, I would suggest we push to complete as much work >>>>> as >>>>>>>>>> possible (from the 0.12 backlog and beyond) by the end of the >>>>> year, >>>>>>>>>> and release as soon as possible in 2019. Of course, anyone is >>>>> welcome >>>>>>>>>> to contribute work that is not found in the 0.12 milestone =) >>>>>>>>>> >>>>>>>>>> Any objections? >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> Wes >>>>>>>>>> On Mon, Dec 10, 2018 at 8:04 AM Andy Grove < >>>>> andygrov...@gmail.com> >>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Cool. I will continue to add primitive operations but I am now >>>>>>> adding this >>>>>>>>>>> in a separate source file to keep it separate from the core >>>>> array >>>>>>> code. >>>>>>>>>>> >>>>>>>>>>> I'm not sure how important it will be to support Rust data >>>>> sources >>>>>>> with >>>>>>>>>>> Gandiva. I can see that each language should be able to >>>>> construct >>>>>>> the >>>>>>>>>>> logical query plan to submit to Gandiva and let Gandiva handle >>>>>>> execution. I >>>>>>>>>>> think the more interesting part is how do we support >>>>>>> language-specific >>>>>>>>>>> lambda functions as part of that logical query plan. Maybe it is >>>>>>> possible >>>>>>>>>>> to compile the lambda down to LLVM (I haven't started learning >>>>>>> about LLVM >>>>>>>>>>> in detail yet so this is wild speculation on my part). Another >>>>>>> option is >>>>>>>>>>> for Gandiva to support calling into shared libraries and that >>>>> maybe >>>>>>> is >>>>>>>>>>> simpler for languages that support building C-native shared >>>>>>> libraries (Rust >>>>>>>>>>> supports this with zero overhead). >>>>>>>>>>> >>>>>>>>>>> Andy. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Sun, Dec 9, 2018 at 11:42 AM Wes McKinney < >>>>> wesmck...@gmail.com> >>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> hi Andy, >>>>>>>>>>>> >>>>>>>>>>>> I can see an argument for having some basic native function >>>>> kernel >>>>>>>>>>>> support in Rust. One of the things that Gandiva has begun is a >>>>>>>>>>>> Protobuf-based serialized representation representation of >>>>>>> projection >>>>>>>>>>>> and filter expressions. In the long run I would like to see a >>>>> more >>>>>>>>>>>> complete relational algebra / logical query plan that can be >>>>>>> submitted >>>>>>>>>>>> for execution. There's complexities, though, such as bridging >>>>>>>>>>>> iteration of data sources written in Rust, say, with a query >>>>> engine >>>>>>>>>>>> written in C++. You would need to provide some kind of a >>>>> callback >>>>>>>>>>>> mechanism for the query engine to request the next chunk of a >>>>>>> dataset >>>>>>>>>>>> to be materialized. >>>>>>>>>>>> >>>>>>>>>>>> It will be interested to see what contributors will be >>>>> motivated >>>>>>>>>>>> enough to build over the next few years. At the end of the day, >>>>>>> Apache >>>>>>>>>>>> projects are do-ocracies. >>>>>>>>>>>> >>>>>>>>>>>> - Wes >>>>>>>>>>>> On Fri, Dec 7, 2018 at 6:22 AM Andy Grove < >>>>> andygrov...@gmail.com> >>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> I've added one PR to the list ( >>>>>>> https://github.com/apache/arrow/pull/3119 >>>>>>>>>>>> ) >>>>>>>>>>>>> to update the project to use Rust 2018 Edition. >>>>>>>>>>>>> >>>>>>>>>>>>> I'm also considering removing one PR from the list and would >>>>> like >>>>>>> to get >>>>>>>>>>>>> opinions here. >>>>>>>>>>>>> >>>>>>>>>>>>> I have a PR (https://github.com/apache/arrow/pull/3033) to >>>>> add >>>>>>> some >>>>>>>>>>>> basic >>>>>>>>>>>>> math and comparison operators to primitive arrays. These are >>>>> baby >>>>>>> steps >>>>>>>>>>>>> towards implementing more query execution capabilities such as >>>>>>>>>>>> projection, >>>>>>>>>>>>> selection, etc but Chao made a good point that other Rust >>>>>>> implementations >>>>>>>>>>>>> don't have these kind of capabilities and I am now wondering >>>>> if >>>>>>> this is a >>>>>>>>>>>>> distraction. We already have Gandiva and the new efforts in >>>>> Ursa >>>>>>> labs and >>>>>>>>>>>>> it would probably make more sense to look at having Rust >>>>> bindings >>>>>>> for the >>>>>>>>>>>>> query execution capabilities there rather than having a >>>>> competing >>>>>>> (and >>>>>>>>>>>> less >>>>>>>>>>>>> capable) implementation in Rust. >>>>>>>>>>>>> >>>>>>>>>>>>> Thoughts? >>>>>>>>>>>>> >>>>>>>>>>>>> Andy. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Thu, Dec 6, 2018 at 8:42 PM paddy horan < >>>>>>> paddyho...@hotmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Other than Andy’s PR below I’m going to try and find time to >>>>>>> work on >>>>>>>>>>>>>> ARROW-3827, I’ll bump it 0.13 if I can’t find the time early >>>>>>> next week. >>>>>>>>>>>>>> There is nothing else in the 0.12 backlog for Rust. It >>>>> would be >>>>>>> nice >>>>>>>>>>>> to >>>>>>>>>>>>>> get the parquet merge in though. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Paddy >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> ________________________________ >>>>>>>>>>>>>> From: Andy Grove <andygrov...@gmail.com> >>>>>>>>>>>>>> Sent: Thursday, December 6, 2018 10:20:48 AM >>>>>>>>>>>>>> To: dev@arrow.apache.org >>>>>>>>>>>>>> Subject: Re: Timeline for Arrow 0.12.0 release >>>>>>>>>>>>>> >>>>>>>>>>>>>> I have PRs pending for all the Rust issues that I want to get >>>>>>> into >>>>>>>>>>>> 0.12.0 >>>>>>>>>>>>>> and would appreciate some reviews so I can go ahead and >>>>> merge: >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://github.com/apache/arrow/pull/3033 (covers >>>>> ARROW-3880 and >>>>>>>>>>>>>> ARROW-3881 >>>>>>>>>>>>>> - add math and comparison operations to primitive arrays) >>>>>>>>>>>>>> https://github.com/apache/arrow/pull/3096 (ARROW-3885 - Rust >>>>>>> release >>>>>>>>>>>>>> process) >>>>>>>>>>>>>> https://github.com/apache/arrow/pull/3111 (ARROW-3838 - CSV >>>>>>> Writer) >>>>>>>>>>>>>> >>>>>>>>>>>>>> With these in place I plan on writing a tutorial for reading >>>>> a >>>>>>> CSV >>>>>>>>>>>> file, >>>>>>>>>>>>>> performing some operations on primitive arrays and writing >>>>> the >>>>>>> output >>>>>>>>>>>> to a >>>>>>>>>>>>>> new CSV file. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I am deferring ARROW-3882 (casting for primitive arrays) to >>>>>>> 0.13.0 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Andy. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Dec 4, 2018 at 7:57 PM Andy Grove < >>>>> andygrov...@gmail.com >>>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'd love to tackle the three related issues for supporting >>>>>>> simple >>>>>>>>>>>>>>> math/comparison operations on primitive arrays and casting >>>>>>> primitive >>>>>>>>>>>>>> arrays >>>>>>>>>>>>>>> but since the change to use Rust specialization feature I'm >>>>> a >>>>>>> bit >>>>>>>>>>>> stuck >>>>>>>>>>>>>> and >>>>>>>>>>>>>>> need some assistance applying the math operations to the >>>>> numeric >>>>>>>>>>>> types >>>>>>>>>>>>>> and >>>>>>>>>>>>>>> not the boolean primitives. I have added a comment to >>>>>>>>>>>>>>> https://github.com/apache/arrow/pull/3033 ... if I can get >>>>> help >>>>>>>>>>>> solving >>>>>>>>>>>>>>> for this PR then I should be able to handle the others. I'll >>>>>>> also do >>>>>>>>>>>> some >>>>>>>>>>>>>>> research and try and figure this out myself. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Andy. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Tue, Dec 4, 2018 at 7:03 PM Wes McKinney < >>>>>>> wesmck...@gmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Andy, Paddy, or other Rust developers -- could you review >>>>> the 6 >>>>>>>>>>>> issues >>>>>>>>>>>>>>>> in TODO in the 0.12 backlog and either assign them or move >>>>>>> them to >>>>>>>>>>>> the >>>>>>>>>>>>>>>> next release if they aren't going to be completed this >>>>> week or >>>>>>> next? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Fri, Nov 30, 2018 at 4:34 PM Wes McKinney < >>>>>>> wesmck...@gmail.com> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> hi folks, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Tomorrow is December 1. The last major Arrow release >>>>> (0.11.0) >>>>>>> took >>>>>>>>>>>>>>>>> place on October 8. Given how much work has happened in >>>>> the >>>>>>>>>>>> project in >>>>>>>>>>>>>>>>> the last ~2 months, I think it would be great to complete >>>>> the >>>>>>> next >>>>>>>>>>>>>>>>> major release before the end-of-year holidays set in. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I've been curating the JIRA backlog the last couple of >>>>> weeks, >>>>>>> and >>>>>>>>>>>> have >>>>>>>>>>>>>>>>> just created a 0.12.0 release wiki page to help us stay >>>>>>> organized >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>> >> https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.12.0+Release >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Given that there are only 3 full working weeks between >>>>> now and >>>>>>>>>>>>>>>>> Christmas, I think we should be in position to cut a >>>>> release >>>>>>> by >>>>>>>>>>>> the >>>>>>>>>>>>>>>>> end of the week of December 10, i.e. by Friday December >>>>> 14. >>>>>>> Not >>>>>>>>>>>> all of >>>>>>>>>>>>>>>>> the TODO issues have to be completed to make the release, >>>>> but >>>>>>> it >>>>>>>>>>>> would >>>>>>>>>>>>>>>>> be good to push to complete as much as possible. Please >>>>> help >>>>>>> by >>>>>>>>>>>>>>>>> reviewing the backlog, and if possible, assigning issues >>>>> to >>>>>>>>>>>> yourself >>>>>>>>>>>>>>>>> that you'd like to pursue in the next 2 weeks. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Let me know if this sounds reasonable, or any concerns. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>>> Wes >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>> >>>>> >> >
