Thanks Adrian, Hari and Shixiong for your inputs. We are using Spark 1.2.1.
Explored Streaming UI, it has very good stats. Thanks for the support - Siva On Tue, Sep 29, 2015 at 5:33 AM, Adrian Tanase <atan...@adobe.com> wrote: > You can also use the REST api introduced in 1.4 – although it’s harder to > parse: > > - jobs from the same batch are not grouped together > - You only get total delay, not scheduling delay > > > From: Hari Shreedharan > Date: Tuesday, September 29, 2015 at 5:27 AM > To: Shixiong Zhu > Cc: Siva, "user@spark.apache.org" > Subject: Re: Monitoring tools for spark streaming > > +1. The Streaming UI should give you more than enough information. > > Thanks, > Hari > > > On Mon, Sep 28, 2015 at 9:55 PM, Shixiong Zhu <zsxw...@gmail.com> wrote: > >> Which version are you using? Could you take a look at the new Streaming >> UI in 1.4.0? >> >> Best Regards, >> Shixiong Zhu >> >> 2015-09-29 7:52 GMT+08:00 Siva <sbhavan...@gmail.com>: >> >>> Hi, >>> >>> Could someone recommend the monitoring tools for spark streaming? >>> >>> By extending StreamingListener we can dump the delay in processing of >>> batches and some alert messages. >>> >>> But are there any Web UI tools where we can monitor failures, see delays >>> in processing, error messages and setup alerts etc. >>> >>> Thanks >>> >>> >> >
