To get the streaming latency I just look at the stats on the application
drivers UI webpage.
I don't know if you can do that programatically, but you could CURL and parse
the page if you had to.
Jeremy Lee BCompSci (Hons)
The Unorthodox Engineers
> On 10 Jun 2014, at 3:36 pm, Yingjun Wu wr
Hi Sean,
Thanks for your reply! So for the first question, any idea for measuring
latency of Spark-Streaming?
Regards,
Yingjun
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-window-processing-tp7234p7300.html
Sent from the Apache Spark Use
To your second question, that is what the 'invFunc' in
reduceByKeyAndWindow() does. If you can supply an "un-reduce" function
the windows can be updated rather than recomputed each time.
On Mon, Jun 9, 2014 at 5:39 AM, Yingjun Wu wrote:
> Dear all,
>
> I just run the window processing job using S
