Hi,
In the spark streaming paper, "slack time" has been suggested for delaying the
batch creation in case of external timestamps. I don't see any such option in
streamingcontext. Is it available in the API?
Also going through the previous posts, queueStream has been suggested for this.
I looked into to queueStream example.
// Create and push some RDDs into Queue
for (i <- 1 to 30) {
rddQueue += ssc.sparkContext.makeRDD(1 to 10)
Thread.sleep(1000)
}
The only thing I am unsure is how to make batches(basic RDD) out of stream
coming on a port.
Regards,
Laeeq