RE: Tranforming flume events using Spark transformation functions

2014-07-22 Thread Sundaram, Muthu X.
I tried to map SparkFlumeEvents to String of RDDs like below. But that map and call are not at all executed. I might be doing this in a wrong way. Any help would be appreciated. flumeStream.foreach(new Function,Void> () { @Override public Void call(JavaRDD eventsData)

Tranforming flume events using Spark transformation functions

2014-07-22 Thread Sundaram, Muthu X.
Hi All, I am getting events from flume using following line. JavaDStream flumeStream = FlumeUtils.createStream(ssc, host, port); Each event is a delimited record. I like to use some of the transformation functions like map and reduce on this. Do I need to convert the JavaDStream to JavaDSt