Re: How do I persist the data after I process the data with Structured streaming...

2016-11-22 Thread shyla deshpande
Has anyone written a custom sink to persist data to Cassandra from structured streaming. Please provide me any link or reference. Thanks On Tue, Nov 22, 2016 at 2:40 PM, Michael Armbrust wrote: > Forgot the link: https://spark.apache.org/docs/latest/structured- > streaming-programming-guide.htm

Re: How do I persist the data after I process the data with Structured streaming...

2016-11-22 Thread Michael Armbrust
We are looking to add a native JDBC sink in Spark 2.2. Until then you can write your own connector using df.writeStream.foreach. On Tue, Nov 22, 2016 at 12:55 PM, shyla deshpande wrote: > Hi, > > Structured streaming works great with Kafka source but I need to persist > the data after processin

Re: How do I persist the data after I process the data with Structured streaming...

2016-11-22 Thread Michael Armbrust
Forgot the link: https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#using-foreach On Tue, Nov 22, 2016 at 2:40 PM, Michael Armbrust wrote: > We are looking to add a native JDBC sink in Spark 2.2. Until then you can > write your own connector using df.writeStream.fo