Re: Getting the batch time of the active batches in spark streaming

2015-11-24 Thread Todd Nist
Hi Abhi, Sorry that was the wrong link should have been the StreamListener, http://spark.apache.org/docs/latest/api/java/org/apache/spark/streaming/scheduler/StreamingListener.html The BatchInfo can be obtained from the event, for example: public void onBatchSubmitted(StreamingListenerBatchSubmi

Re: Getting the batch time of the active batches in spark streaming

2015-11-24 Thread Todd Nist
Hi Abhi, You should be able to register a org.apache.spark.streaming.scheduler.StreamListener. There is an example here that may help: https://gist.github.com/akhld/b10dc491aad1a2007183 and the spark api docs here, http://spark.apache.org/docs/latest/api/java/org/apache/spark/scheduler/SparkListe

Getting the batch time of the active batches in spark streaming

2015-11-24 Thread Abhishek Anand
Hi , I need to get the batch time of the active batches which appears on the UI of spark streaming tab, How can this be achieved in Java ? BR, Abhi