The streaming query should keep polling data from Kafka. When the query was stopped, did you see any exception?
Best Regards, Shixiong Zhu Databricks Inc. shixi...@databricks.com databricks.com [image: http://databricks.com] <http://databricks.com/> <https://databricks.com/sparkaisummit/north-america> On Tue, May 15, 2018 at 6:25 PM, karthikjay <aswin8...@gmail.com> wrote: > Hi all, > > We are running into a scenario where the structured streaming job is > exiting > after a while specifically when the Kafka topic is not getting any data. > From the job logs, I see this connections.max.idle.ms = 540000. Does that > mean the spark readstream will close when it does not get data for 540000 > milliseconds ? If yes, how do I override this ? > > > > -- > Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
