Dear all, I just run the window processing job using Spark-Streaming, and I have two questions. First, how can I measure the latency of Spark-Streaming? Is there any APIs that I can call directly? Second, is it true that the latency of Spark-Streaming grows linearly with the window size? It seems that the computation model of Spark-Streaming does not directly support incremental computation. For example, to calculate the sum of five continuous numbers from 1 to infinite, the first computation should accumulate 1, 2, 3, 4, 5, and the second should accumulate 2, 3, 4, 5, 6. That is, the result we obtained from the first computation cannot directly be applied to the next computation round, correct?
Thanks for your kind attention to this message. Regards, Yingjun -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-window-processing-tp7234.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
