Re: Scala examples for Spark do not work as written in documentation

2014-06-20 Thread Will Benton
Hey, sorry to reanimate this thread, but just a quick question: why do the examples (on http://spark.apache.org/examples.html) use "spark" for the SparkContext reference? This is minor, but it seems like it could be a little confusing for people who want to run them in the shell and need to ch

Re: Scala examples for Spark do not work as written in documentation

2014-06-20 Thread Patrick Wendell
Those are pretty old - but I think the reason Matei did that was to make it less confusing for brand new users. `spark` is actually a valid identifier because it's just a variable name (val spark = new SparkContext()) but I agree this could be confusing for users who want to drop into the shell. O

Current status of Sparrow

2014-06-20 Thread Liquan Pei
Hi What is the current status of Sparrow integration with Spark? I would like to integrate Sparrow with Spark 1.0 on a 100 node cluster. Any suggestions? Thanks a lot for your help! Liquan