Re: [VOTE] Apache Spark 2.1.0 (RC1)

We still have several blockers for 2.1, so I imagine at least one will mean this won't be the final RC: SPARK-18318 ML, Graph 2.1 QA: API: New Scala APIs, docs SPARK-18319 ML, Graph 2.1 QA: API: Experimental, DeveloperApi, final, sealed audit SPARK-18326 SparkR 2.1 QA: New R APIs and API docs SPAR

Re: [VOTE] Apache Spark 2.1.0 (RC1)

I'll send a -1 because of SPARK-18546. Haven't looked at anything else yet. On Mon, Nov 28, 2016 at 5:25 PM, Reynold Xin wrote: > Please vote on releasing the following candidate as Apache Spark version > 2.1.0. The vote is open until Thursday, December 1, 2016 at 18:00 UTC and > passes if a majo

Please add me

Hi, I am interested to contribute code on Spark. Could you please add me into the mailing list / DL. Thanks, *Srinivas Potluri*

Can't read tables written in Spark 2.1 in Spark 2.0 (and earlier)

Hello, When I try to read from a Hive table created by Spark 2.1 in Spark 2.0 or earlier, I get an error: java.lang.ClassNotFoundException: Failed to load class for data source: hive. Is there a way to get previous versions of Spark to read tables written with Spark 2.1? Cheers, Michael

Re: Can't read tables written in Spark 2.1 in Spark 2.0 (and earlier)

This is not an issue with all tables created in Spark 2.1, though I'm not sure why some work and some do not. I have found that a table created as such sql("create table test stored as parquet as select 1") in Spark 2.1 cannot be read in previous versions of Spark. Michael > On Nov 29, 2016,

Question about spark.mllib.GradientDescent

Hi devs: I think it's unnecessary to use c1._1 += c2.1 in combOp operation, I think it's the same if we use c1._1+c2._1, see the code below : in GradientDescent.scala val (gradientSum, lossSum, miniBatchSize) = data.sample(false, miniBatchFraction, 42 + i) .treeAggregate((BDV.zeros[D

Spark-9487, Need some insight

Hello Spark dev community, I took this the following jira item (https://github.com/apache/spark/pull/15848) and am looking for some general pointers, it seems that I am running into issues where things work successfully doing local development on my macbook pro but fail on jenkins for a multiti

Re: How is the order ensured in the jdbc relation provider when inserting data from multiple executors

Hi all, To explain the scenario a bit more. We need to retain the order when writing to the RDBMS tables. The way we found was to execute the DB Write *job* for each partition which is really costly. One reason being that the partition count is really high( 200) and it seems we cannot control the

Proposal for SPARK-18278

Hi everyone, Kubernetes is a technology that is a key player in the cluster computing world. Currently, running Spark applications on Kubernetes requires deploying a standalone Spark cluster on the Kubernetes cluster, and then running the jobs against the standalone Spark cluster. However, ther

Re: Please add me

Hi Srinivas, You can subscribe the mail list in the community page by yourself http://spark.apache.org/community.html On Tue, Nov 29, 2016 at 9:59 AM, Srinivas Potluri wrote: > Hi, > > I am interested to contribute code on Spark. Could you please add me into > the mailing list / DL. > > Thanks,