Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-19 Thread Reynold Xin
The vote passed with the following +1 and -1: +1 Reynold Xin* Sean Owen* Dongjoon Hyun Xiao Li Herman van Hövell tot Westerflier Joseph Bradley* Liwei Lin Denny Lee Holden Karau Adam Roberts vaquar khan 0/+1 (not sure what this means but putting it here just in case) Felix Cheung -1 Franklyn

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-19 Thread Nicholas Chammas
Since it’s not a regression from 2.0 (I believe the same issue affects both 2.0 and 2.1) it doesn’t merit a -1 vote according to the voting guidelines. Of course, it would be nice if we could fix the various optimizer issues that all seem to have a workaround that involves persist() (another one i

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-19 Thread Franklyn D'souza
-1 https://issues.apache.org/jira/browse/SPARK-18589 hasn't been resolved by this release and is a blocker in our adoption of spark 2.0. I've updated the issue with some steps to reproduce the error. On Mon, Dec 19, 2016 at 4:37 AM, Sean Owen wrote: > PS, here are the open issues for 2.1.0. Forg

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-19 Thread Sean Owen
PS, here are the open issues for 2.1.0. Forgot this one. No Blockers, but one "Critical": SPARK-16845 org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB SPARK-18669 Update Apache docs regard watermarking in Structured Streaming SPARK-18894 Event time wa

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-18 Thread Felix Cheung
ent: Sunday, December 18, 2016 2:33 PM Subject: Re: [VOTE] Apache Spark 2.1.0 (RC5) To: Adam Roberts mailto:arobe...@uk.ibm.com>> Cc: Denny Lee mailto:denny.g@gmail.com>>, Holden Karau mailto:hol...@pigscanfly.ca>>, Liwei Lin mailto:lwl...@gmail.com>>, mailto:dev@spark.apach

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-18 Thread vaquar khan
slowdowns for q7, q39a, q43, q52, > q57, q89. Five iterations, average times compared, only changing which > version of Spark we're using > > > > From: Holden Karau > To:Denny Lee , Liwei Lin , > "dev@spark.apache.org" > Date:18/1

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-18 Thread Adam Roberts
q89. Five iterations, average times compared, only changing which version of Spark we're using From: Holden Karau To: Denny Lee , Liwei Lin , "dev@spark.apache.org" Date: 18/12/2016 20:05 Subject: Re: [VOTE] Apache Spark 2.1.0 (RC5) +1 (non-binding) - check

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-18 Thread Holden Karau
gt; > For R we have a license field in the DESCRIPTION, and this is standard > practice (and requirement) for R packages. > > > > > > > > https://cran.r-project.org/doc/manuals/R-exts.html#Licensing > > > > > > > > -

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-18 Thread Denny Lee
gt; > -- > *From:* Sean Owen > *Sent:* Friday, December 16, 2016 9:57:15 AM > *To:* Reynold Xin; dev@spark.apache.org > *Subject:* Re: [VOTE] Apache Spark 2.1.0 (RC5) > > (If you have a template for these emails, maybe update it to use https > links. T

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-17 Thread Liwei Lin
t;>>>> https://cran.r-project.org/doc/manuals/R-exts.html#Licensing >>>>> >>>>> -- >>>>> *From:* Sean Owen >>>>> *Sent:* Friday, December 16, 2016 9:57:15 AM >>>>> *To:* Reynold Xin; dev@

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-16 Thread Yuming Wang
ld in the DESCRIPTION, and this is standard >>>> practice (and requirement) for R packages. >>>> >>>> https://cran.r-project.org/doc/manuals/R-exts.html#Licensing >>>> >>>> -------------- >>>> *From:* Sean Owen &g

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-16 Thread Joseph Bradley
Sean Owen >>> *Sent:* Friday, December 16, 2016 9:57:15 AM >>> *To:* Reynold Xin; dev@spark.apache.org >>> *Subject:* Re: [VOTE] Apache Spark 2.1.0 (RC5) >>> >>> (If you have a template for these emails, maybe update it to use https >>> link

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-16 Thread Herman van Hövell tot Westerflier
cran.r-project.org/doc/manuals/R-exts.html#Licensing >> >> -- >> *From:* Sean Owen >> *Sent:* Friday, December 16, 2016 9:57:15 AM >> *To:* Reynold Xin; dev@spark.apache.org >> *Subject:* Re: [VOTE] Apache Spark 2.1.0 (RC5) >> >

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-16 Thread Xiao Li
gt; *From:* Sean Owen > *Sent:* Friday, December 16, 2016 9:57:15 AM > *To:* Reynold Xin; dev@spark.apache.org > *Subject:* Re: [VOTE] Apache Spark 2.1.0 (RC5) > > (If you have a template for these emails, maybe update it to use https > links. They work for apache.org domains. After a

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-16 Thread Felix Cheung
@spark.apache.org Subject: Re: [VOTE] Apache Spark 2.1.0 (RC5) (If you have a template for these emails, maybe update it to use https links. They work for apache.org<http://apache.org> domains. After all we are asking people to verify the integrity of release artifacts, so it might as well be

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-16 Thread Dongjoon Hyun
RC5 is also tested on CentOS 6.8, OpenJDK 1.8.0_111, R 3.3.2 with profiles `-Pyarn -Phadoop-2.7 -Pkinesis-asl -Phive -Phive-thriftserver -Psparkr`. BTW, there still exist five on-going issues in JIRA (with target version 2.1.0). 1. SPARK-16845 org.apache.spark.sql.catalyst.expressions.Generate

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-16 Thread Sean Owen
(If you have a template for these emails, maybe update it to use https links. They work for apache.org domains. After all we are asking people to verify the integrity of release artifacts, so it might as well be secure.) (Also the new archives use .tar.gz instead of .tgz like the others. No big de

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-16 Thread Holden Karau
Thanks for the specific mention of the new PySpark packaging Shivaram, For *nix (Linux, Unix, OS X, etc.) Python users interested in helping test the new artifacts you can do as follows: Setup PySpark with pip by: 1. Download the artifact from http://home.apache.org/~pwendell/spark-releases/spar

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-15 Thread Reynold Xin
I'm going to start this with a +1! On Thu, Dec 15, 2016 at 9:42 PM, Shivaram Venkataraman < shiva...@eecs.berkeley.edu> wrote: > In addition to usual binary artifacts, this is the first release where > we have installable packages for Python [1] and R [2] that are part of > the release. I'm inc

Re: [VOTE] Apache Spark 2.1.0 (RC5)

2016-12-15 Thread Shivaram Venkataraman
In addition to usual binary artifacts, this is the first release where we have installable packages for Python [1] and R [2] that are part of the release. I'm including instructions to test the R package below. Holden / other Python developers can chime in if there are special instructions to test