FileNotFoundException with _temporary in the name

Hi Spark devs, Several people on the mailing list have seen issues with FileNotFoundExceptions related to _temporary in the name. I've personally observed this several times, as have a few of my coworkers on various Spark clusters. Any ideas what might be going on? I've collected the various st

Re: Using mllib-1.1.0-SNAPSHOT on Spark 1.0.1

I figured out the issuethe driver memory was at 512 MB and for our datasets, the following code needed more memory... // Materialize usersOut and productsOut. usersOut.count() productsOut.count() Thanks. Deb On Sat, Aug 9, 2014 at 6:12 PM, Debasish Das wrote: > Actually nope it did not

Re: Spark testsuite error for hive 0.13.

Problem solved by a walkaround with create database and use database. -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Spark-testsuite-error-for-hive-0-13-tp7807p7819.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. -

Re: Contributing to MLlib: Proposal for Clustering Algorithms

Hi all, I wanted to follow up. I have a prototype for an optimized version of hierarchical k-means. I wanted to get some feedback on my apporach. Jeremy's implementation splits the largest cluster in each round. Is it better to do it that way or to split each cluster in half? Are there are an

Re: fair scheduler

@Crystal You can use spark on yarn. Yarn have fair scheduler,modified yarn-site.xml. 发自我的 iPad > 在 2014年8月11日，6:49，Matei Zaharia 写道： > > Hi Crystal, > > The fair scheduler is only for jobs running concurrently within the same > SparkContext (i.e. within an application), not for separate appli

Re: [SPARK-2878] Kryo serialisation with custom Kryo registrator failing

I've submitted a work-in-progress pull request for this issue that I'd like feedback on. See https://github.com/apache/spark/pull/1890 . I've also submitted a pull request for the related issue that the exceptions hit when trying to use a custom kryo registrator are being swallowed: https://github

Compie error with XML elements

When compiling the master checkout of spark. The Intellij compile fails with: Error:(45, 8) not found: value $scope ^ which is caused by HTML elements in classes like HistoryPage.scala: val content = ... How can I compile these classes that have html node ele

Re: Spark Avro Generation

Thanks very much that helps, not having to generate the entire build. On Mon, Aug 11, 2014 at 6:09 PM, Ron Gonzalez wrote: > If you don't want to build the entire thing, you can also do > > mvn generate-sources in externals/flume-sink > > Thanks, > Ron > > Sent from my iPhone > > > On Aug 11, 2