branch-1.0 cut

Hey All, In accordance with the scheduled window for the release I've cut a 1.0 branch. Thanks a ton to everyone for being so active in reviews during the last week. In the last 7 days we've merged 66 new patches, and every one of them has undergone thorough peer-review. Tons of committers have be

Re: branch-1.0 cut

Wow, great work. Very impressive sticking to the schedule! On Wed, Apr 9, 2014 at 2:31 AM, Patrick Wendell wrote: > Hey All, > > In accordance with the scheduled window for the release I've cut a 1.0 > branch. Thanks a ton to everyone for being so active in reviews during the > last week. In th

Spark 0.9.1 released

Hi everyone, We have just posted Spark 0.9.1, which is a maintenance release with bug fixes, performance improvements, better stability with YARN and improved parity of the Scala and Python API. We recommend all 0.9.0 users to upgrade to this stable release. This is the first release since Spark

Re: Spark 0.9.1 released

A small additional note: Please use the direct download links in the Spark Downloads page. The Apache mirrors take a day or so to sync from the main repo, so may not work immediately. TD On Wed, Apr 9, 2014 at 2:54 PM, Tathagata Das wrote: > Hi everyone,

Re: Contributing to Spark

Another starter question which probably should have asked before is what is the most efficient way to iterate quickly on dev/test. I am currently using a local cluster (via vagrant and shared folders) and also spark-shell. Sujeet On Tue, Apr 8, 2014 at 9:50 AM, Michael Ernest wrote: > Ha ha! n

Re: Spark 0.9.1 released

Thanks TD for managing this release, and thanks to everyone who contributed! Matei On Apr 9, 2014, at 2:59 PM, Tathagata Das wrote: > A small additional note: Please use the direct download links in the Spark > Downloads page. The Apache mirrors take a day or so to sync from the main > repo,

Re: Contributing to Spark

Usually you can just run Spark in local mode on a single machine for most dev/testing. If you want to simulate a cluster locally using multiple Spark worker processes, you can use the undocumented local cluster mode, e.g. local-cluster[2,1,512] this launches two worker processes, each with one c

Re: Spark 0.9.1 released

Thanks Nick for pointing that out! I have updated the release notes. But I see the new operations like repartition in the latest PySpark RDD docs. Maybe refresh the page couple of

Updating all references to github.com/apache/incubator-spark on spark website

Since http://github.com/apache/incubator-spark and any links underneath it now return 404, I propose we do a global search and replace to change all instances to remove "incubator-", including those in docs/0.8.0 docs/0.8.1 and docs/0.9.0. I'm happy to do this. Any discussion before I do? Andy