Re: [discuss] Spark 2.x release cadence

2016-09-29 Thread Weiqing Yang
Sorry. I think I just replied to the wrong thread. :( WQ On Thu, Sep 29, 2016 at 10:58 AM, Weiqing Yang wrote: > +1 (non binding) > > > > RC4 is compiled and tested on the system: CentOS Linux release > 7.0.1406 / openjdk 1.8.0_102 / R 3.3.1 > > All tests passed. > > > > ./build/mvn -Pyarn -P

Re: [discuss] Spark 2.x release cadence

2016-09-29 Thread Weiqing Yang
+1 (non binding) RC4 is compiled and tested on the system: CentOS Linux release 7.0.1406 / openjdk 1.8.0_102 / R 3.3.1 All tests passed. ./build/mvn -Pyarn -Phadoop-2.7 -Pkinesis-asl -Phive -Phive-thriftserver -Dpyspark -Dsparkr -DskipTests clean package ./build/mvn -Pyarn -Phadoop-2.7 -Pk

Re: [discuss] Spark 2.x release cadence

2016-09-29 Thread Cody Koeninger
Regarding documentation debt, is there a reason not to deploy documentation updates more frequently than releases? I recall this used to be the case. On Wed, Sep 28, 2016 at 3:35 PM, Joseph Bradley wrote: > +1 for 4 months. With QA taking about a month, that's very reasonable. > > My main ask (

Re: [discuss] Spark 2.x release cadence

2016-09-28 Thread Joseph Bradley
+1 for 4 months. With QA taking about a month, that's very reasonable. My main ask (especially for MLlib) is for contributors and committers to take extra care not to delay on updating the Programming Guide for new APIs. Documentation debt often collects and has to be paid off during QA, and a l

Re: [discuss] Spark 2.x release cadence

2016-09-28 Thread Tom Graves
+1 to 4 months. Tom On Tuesday, September 27, 2016 2:07 PM, Reynold Xin wrote: We are 2 months past releasing Spark 2.0.0, an important milestone for the project. Spark 2.0.0 deviated (took 6 month from the regular release cadence we had for the 1.x line, and we never explicitly discu

Re: [discuss] Spark 2.x release cadence

2016-09-27 Thread Felix Cheung
+1 on longer release cycle at schedule and more maintenance releases. _ From: Mark Hamstra mailto:m...@clearstorydata.com>> Sent: Tuesday, September 27, 2016 2:01 PM Subject: Re: [discuss] Spark 2.x release cadence To: Reynold Xin mailto:r...@databricks.co

Re: [discuss] Spark 2.x release cadence

2016-09-27 Thread Mark Hamstra
+1 And I'll dare say that for those with Spark in production, what is more important is that maintenance releases come out in a timely fashion than that new features are released one month sooner or later. On Tue, Sep 27, 2016 at 12:06 PM, Reynold Xin wrote: > We are 2 months past releasing Spa

Re: [discuss] Spark 2.x release cadence

2016-09-27 Thread Sean Owen
+1 -- I think the minor releases were taking more like 4 months than 3 months anyway, and it was good for the reasons you give. This reflects reality and is a good thing. All the better if we then can more comfortably really follow the timeline. On Tue, Sep 27, 2016 at 3:06 PM, Reynold Xin wrote:

Re: [discuss] Spark 2.x release cadence

2016-09-27 Thread Shivaram Venkataraman
+1 I think having a 4 month window instead of a 3 month window sounds good. However I think figuring out a timeline for maintenance releases would also be good. This is a common concern that comes up in many user threads and it'll be better to have some structure around this. It doesn't need to be

[discuss] Spark 2.x release cadence

2016-09-27 Thread Reynold Xin
We are 2 months past releasing Spark 2.0.0, an important milestone for the project. Spark 2.0.0 deviated (took 6 month from the regular release cadence we had for the 1.x line, and we never explicitly discussed what the release cadence should look like for 2.x. Thus this email. During Spark 1.x, r