cannot call explain or show on dataframe in structured streaming addBatch dataframe

[assaf.mendelson]

Hi all,
I am playing around with structured streaming and looked at the code for 
ConsoleSink.

I see the code has:


data.sparkSession.createDataFrame(
    data.sparkSession.sparkContext.parallelize(data.collect()), data.schema)
    .show(numRowsToShow, isTruncated)
}

I was wondering why it does not do data directly? Why the collect and 
parallelize?


Thanks,
              Assaf.





--
View this message in context: 
http://apache-spark-developers-list.1001551.n3.nabble.com/cannot-call-explain-or-show-on-dataframe-in-structured-streaming-addBatch-dataframe-tp21792.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.