Hi Is there a way to write all RDDs in a DStream to the same file? I tried this and got an empty file. I think it's bc the file is not closed i.e. ESMinibatchFunctions.writer.close() executes before the stream is created.
Here's my code myStream.foreachRDD(rdd => { rdd.foreach(x => { ESMinibatchFunctions.writer.append(rdd.collect()(0).toString()+" the data ") }) // localRdd = localRdd.union(rdd) // localArray = localArray ++ rdd.collect() } ) ESMinibatchFunctions.writer.close() object ESMinibatchFunctions { val writer = new PrintWriter("c:/delme/exxx.txt") }