Re: PySpark 2.0 Structured Streaming Question

2016-07-20 Thread Tathagata Das
foreachWriter is not currently available in the python. we dont have a clear plan yet on when foreachWriter will be available in Python. On Wed, Jul 20, 2016 at 1:22 PM, A.W. Covert III wrote: > Hi All, > > I've been digging into spark 2.0, I have some streaming jobs running well > on YARN, and

PySpark 2.0 Structured Streaming Question

2016-07-20 Thread A.W. Covert III
Hi All, I've been digging into spark 2.0, I have some streaming jobs running well on YARN, and I'm working on some Spark Structured Streaming jobs now. I have a couple of jobs I'd like to move to Structured Streaming with the `foreachWriter` but it's not available in PySpark yet. Is it just becau