Hi , Thank you for your reply ,
Yes , I confirm , the system is stuck after job 352 finishes and before job 353 starts (shows up in the UI) I will start job again and will take jstack if I can reproduce the problem Thanks, Snehal On 2 September 2015 at 13:34, Tathagata Das <t...@databricks.com> wrote: > Can you jstack into the driver and see what is process doing after job 352? > > Also to confirm, the system is stuck after job 352 finishes, and before > job 353 starts (shows up in the UI), right? > > TD > > On Wed, Sep 2, 2015 at 12:55 PM, Snehal Nagmote <nagmote.sne...@gmail.com> > wrote: > >> Hi All, >> >> I have spark job where I read data from Kafka every 5 seconds interval >> and query Cassandra based on Kafka data using spark Cassandra Connector , >> >> I am using spark 1.4 , Often the batch gets stuck in processing after job >> Id 352 . Spark takes long time to spawn job 353 where it reads from >> Cassandra . >> >> I want to understand since job processing time are in seconds, what else >> contribute to processing time ? >> >> What could be the potential issues which is adding processing time here ? >> Any inputs would be helpful. >> >> >> >> >> [image: Inline images 2] >> >> Thanks, >> Snehal >> > >
