Great success! I was able to get output to the driver console by changing the construction of the Streaming Spark Context from:
val ssc = new StreamingContext("local" /**TODO change once a cluster is up **/, "AppName", Seconds(1)) to: val ssc = new StreamingContext("local[2]" /**TODO change once a cluster is up **/, "AppName", Seconds(1)) I found something that tipped me off that this might work by digging through this mailing list. On Sun, Jul 13, 2014 at 2:00 PM, Walrus theCat <walrusthe...@gmail.com> wrote: > More strange behavior: > > lines.foreachRDD(x => println(x.first)) // works > lines.foreachRDD(x => println((x.count,x.first))) // no output is printed > to driver console > > > > > On Sun, Jul 13, 2014 at 11:47 AM, Walrus theCat <walrusthe...@gmail.com> > wrote: > >> >> Thanks for your interest. >> >> lines.foreachRDD(x => println(x.count)) >> >> And I got 0 every once in a while (which I think is strange, because >> lines.print prints the input I'm giving it over the socket.) >> >> >> When I tried: >> >> lines.map(_->1).reduceByKey(_+_).foreachRDD(x => println(x.count)) >> >> I got no count. >> >> Thanks >> >> >> On Sun, Jul 13, 2014 at 11:34 AM, Tathagata Das < >> tathagata.das1...@gmail.com> wrote: >> >>> Try doing DStream.foreachRDD and then printing the RDD count and further >>> inspecting the RDD. >>> On Jul 13, 2014 1:03 AM, "Walrus theCat" <walrusthe...@gmail.com> >>> wrote: >>> >>>> Hi, >>>> >>>> I have a DStream that works just fine when I say: >>>> >>>> dstream.print >>>> >>>> If I say: >>>> >>>> dstream.map(_,1).print >>>> >>>> that works, too. However, if I do the following: >>>> >>>> dstream.reduce{case(x,y) => x}.print >>>> >>>> I don't get anything on my console. What's going on? >>>> >>>> Thanks >>>> >>> >> >