Re: [VOTE] Release Apache Spark 1.4.0 (RC3)

Hi everyone, I think there's a blocker on PySpark the "when" functions in python seems to be broken but the Scala API seems fine. Here's a snippet demonstrating that with Spark 1.4.0 RC3 : In [*1*]: df = sqlCtx.createDataFrame([(1, "1"), (2, "2"), (1, "2"), (1, "2")], ["key", "value"]) In [*2*]:

Re: [VOTE] Release Apache Spark 1.4.0 (RC3)

Hi Patrick, Thanks for clarifying. No issues with functionality. +1 (non-binding) Thanks Bobby On Mon, Jun 1, 2015 at 9:41 PM, Patrick Wendell wrote: > Hey Bobby, > > Those are generic warnings that the hadoop libraries throw. If you are > using MapRFS they shouldn't matter si

Re: [VOTE] Release Apache Spark 1.4.0 (RC3)

Hey Bobby, Those are generic warnings that the hadoop libraries throw. If you are using MapRFS they shouldn't matter since you are using the MapR client and not the default hadoop client. Do you have any issues with functionality... or was it just seeing the warnings that was the concern? Thanks

Re: [VOTE] Release Apache Spark 1.4.0 (RC3)

Hive Context works on RC3 for Mapr after adding spark.sql.hive.metastore.sharedPrefixes as suggested in SPARK-7819 . However, there still seems to be some other issues with native libraries, i get below warning WARN NativeCodeLoader: Unable to load

Re: [VOTE] Release Apache Spark 1.4.0 (RC3)

I get a bunch of failures in VersionSuite with build/test params "-Pyarn -Phive -Phadoop-2.6": - success sanity check *** FAILED *** java.lang.RuntimeException: [download failed: org.jboss.netty#netty;3.2.2.Final!netty.jar(bundle), download failed: commons-net#commons-net;3.1!commons-net.jar]

Re: [VOTE] Release Apache Spark 1.4.0 (RC3)

HiveContext.scala:378) >> at >> org.apache.spark.sql.SQLContext$QueryExecution.assertAnalyzed(SQLContext.scala:901) >> at org.apache.spark.sql.DataFrame.(DataFrame.scala:134) >> at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51) >> at org.apac

Re: [VOTE] Release Apache Spark 1.4.0 (RC3)

dera.com>; *Date: * Mon, Jun 1, 2015 07:34 AM *To: * "Krishna Sankar" <mailto:ksanka...@gmail.com>; *Cc: * "Patrick Wendell" <mailto:pwend...@gmail.com>; "dev@spark.apache.org" <mailto:dev@spark.apache.org> <mailto:dev@

Re: [VOTE] Release Apache Spark 1.4.0 (RC3)

.(DataFrame.scala:134) >> at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51) >> at org.apache.spark.sql.SQLContext.createDataFrame(SQLContext.scala:474) >> at org.apache.spark.sql.SQLContext.createDataFrame(SQLContext.scala:456) >> at >> o

Re: [VOTE] Release Apache Spark 1.4.0 (RC3)

SQLContext.scala:474) > at org.apache.spark.sql.SQLContext.createDataFrame(SQLContext.scala:456) > at > org.apache.spark.sql.SQLContext$implicits$.intRddToDataFrameHolder(SQLContext.scala:345) > > Thanks, > Peter Rudenko > > On 2015-06-01 05:04, Guoqiang Li wrote: >

Re: [VOTE] Release Apache Spark 1.4.0 (RC3)

lder(SQLContext.scala:345) | Thanks, Peter Rudenko On 2015-06-01 05:04, Guoqiang Li wrote: +1 (non-binding) ------ Original ---------- *From: * "Sandy Ryza";; *Date: * Mon, Jun 1, 2015 07:34 AM *To: * "Krishna Sankar"; *Cc: * "Patrick Wendell&quo

Re: [VOTE] Release Apache Spark 1.4.0 (RC3)

+1 (non-binding) -- Original -- From: "Sandy Ryza";; Date: Mon, Jun 1, 2015 07:34 AM To: "Krishna Sankar"; Cc: "Patrick Wendell"; "dev@spark.apache.org"; Subject: Re: [VOTE] Release Apache Spark 1.4.0 (RC3) +1

Re: [VOTE] Release Apache Spark 1.4.0 (RC3)

+1 (non-binding) Launched against a pseudo-distributed YARN cluster running Hadoop 2.6.0 and ran some jobs. -Sandy On Sat, May 30, 2015 at 3:44 PM, Krishna Sankar wrote: > +1 (non-binding, of course) > > 1. Compiled OSX 10.10 (Yosemite) OK Total time: 17:07 min > mvn clean package -Pyarn

Re: [VOTE] Release Apache Spark 1.4.0 (RC3)

+1 (non-binding, of course) 1. Compiled OSX 10.10 (Yosemite) OK Total time: 17:07 min mvn clean package -Pyarn -Dyarn.version=2.6.0 -Phadoop-2.4 -Dhadoop.version=2.6.0 -DskipTests 2. Tested pyspark, mlib - running as well as compare results with 1.3.1 2.1. statistics (min,max,mean,Pearson,Spe

Re: [VOTE] Release Apache Spark 1.4.0 (RC3)

Mike, The broken Configuration link can be fixed if you add a missing dash '-' on the first line in docs/configuration.md and run 'jekyll build'. https://github.com/apache/spark/pull/6513 On Fri, May 29, 2015 at 6:38 PM, Mike Ringenburg wrote: > The Configuration link on the docs appears to b

Re: [VOTE] Release Apache Spark 1.4.0 (RC3)

The Configuration link on the docs appears to be broken. Mike On May 29, 2015, at 4:41 PM, Patrick Wendell mailto:pwend...@gmail.com>> wrote: Please vote on releasing the following candidate as Apache Spark version 1.4.0! The tag to be voted on is v1.4.0-rc3 (commit dd109a8): https://git-wip