答复: [VOTE] Release Apache Spark 1.4.0 (RC4)

+1 Tested with building with Hadoop 2.7.0 and running with tests: WordCount in yarn-client/yarn-cluster mode works fine; Basic sql queries are passed; “spark.sql.autoBroadcastJoinThreshold” works fine; Thrift Server is fine; Running streaming with kafka is good; External shuffle in YARN mode is f

About akka used in spark

Hi guys, I see group id of akka used in spark is "org.spark-project.akka". What is its difference with the typesafe one? What is its version? And where can we get the source code? Regards.

Re: Stages with non-arithmetic numbering & Timing metrics in event logs

Ahhh---forgive my typo: what I mean is, (t2 - t1) >= (t_ser + t_deser + t_exec) is satisfied, empirically. On 6/10/15, Mike Hynes <91m...@gmail.com> wrote: > Hi Imran, > > Thank you for your email. > > In examing the condition (t2 - t1) < (t_ser + t_deser + t_exec), I > have found it to be true, a

Re: Stages with non-arithmetic numbering & Timing metrics in event logs

Hi Imran, Thank you for your email. In examing the condition (t2 - t1) < (t_ser + t_deser + t_exec), I have found it to be true, although I have not included the t_{wait_for_read} in this, since it is---so far as I can tell---been either zero or negligible compared to the task time. Thanks, Mike

Re: Spark on Mesos vs Yarn

Hi Nik, Bharath is mostly referring to Spark commiters in this thread. Tim On Tue, Jun 9, 2015 at 9:51 PM, Niklas Nielsen wrote: > Hi Bharath (and rest of Spark dev list!), > > Just a small shout out: I am a Apache Mesos Committer and would love to help > out with anything you need to get this

Re: Spark on Mesos vs Yarn

Hi Bharath (and rest of Spark dev list!), Just a small shout out: I am a Apache Mesos Committer and would love to help out with anything you need to get this going. Cheers, Nik On 9 June 2015 at 21:10, Bharath Ravi Kumar wrote: > All, > > Despite the common origin of spark & mesos, the stabili

Re: Spark on Mesos vs Yarn

All, Despite the common origin of spark & mesos, the stability and adoption of mesos, and the age of the spark-mesos binding, I find the mesos support less mature, with fundamental shortcomings (like framework auth ) remaining unresolved. If there'

Re: [VOTE] Release Apache Spark 1.4.0 (RC4)

+1 tested /w OS X + deployed one of our streaming apps onto a staging yarn cluster. Sean > On Jun 2, 2015, at 9:54 PM, Patrick Wendell wrote: > > Please vote on releasing the following candidate as Apache Spark version > 1.4.0! > > The tag to be voted on is v1.4.0-rc3 (commit 22596c5): > htt

Re: GraphX: New graph operator

i have a pull request about this issue. https://github.com/apache/spark/pull/6685 the union operation of two graph is useful in practice. And it’s necessary to provide operation on the Graph level. > On 3 Jun, 2015, at 2:58 pm, Reynold Xin wrote: >

Recreating JIRA SPARK-8142

Hi All We are having some trouble with: sparkConf.set("spark.driver.userClassPathFirst","true"); sparkConf.set("spark.executor.userClassPathFirst","true"); and would appreciate some independent verification. The issue comes down to this: Spark 1.3.1 hadoop 2.6 is deployed on the cluster. In my