Hi
Is there a way to write all RDDs in a DStream to the same file?
I tried this and got an empty file. I think it's bc the file is not closed i.e.
ESMinibatchFunctions.writer.close() executes before the stream is created.
Here's my code
myStream.foreachRDD(rdd => {
rdd.foreach(x => {
ESMinibatchFunctions.writer.append(rdd.collect()(0).toString()+" the
data ") })
// localRdd = localRdd.union(rdd)
// localArray = localArray ++ rdd.collect()
} )
ESMinibatchFunctions.writer.close()
object ESMinibatchFunctions {
val writer = new PrintWriter("c:/delme/exxx.txt")
}