println does not execute in the driver since you are executing it on elements of the RDD. It executes in an executor, which can happen to execute in-process in local mode. In general you should not expect this to print results in the driver.
On Sun, May 17, 2015 at 10:01 AM, xiaohe lan <[email protected]> wrote: > Hi, > > When I start spark shell by passing yarn to master option, println does not > print elements in RDD: > > bash-4.1$ spark-shell --master yarn > 15/05/17 01:50:08 WARN NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > Welcome to > ____ __ > / __/__ ___ _____/ /__ > _\ \/ _ \/ _ `/ __/ '_/ > /___/ .__/\_,_/_/ /_/\_\ version 1.3.1 > /_/ > > Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java > 1.8.0_45) > Type in expressions to have them evaluated. > Type :help for more information. > Spark context available as sc. > SQL context available as sqlContext. > > scala> val lines = sc.parallelize(List("hello world", "hi")) > lines: org.apache.spark.rdd.RDD[String] = ParallelCollectionRDD[0] at > parallelize at <console>:21 > > scala> lines.first() > res1: String = hello world > > scala> lines.foreach(println) > > scala> > > If I start spark shell in local mode, the elements are printed. What's the > difference here ? > > Thanks, > Xiaohe --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
