Hello!
I'd like to ask if there is any reason to return *type *when calling
*dataframe.unpersist*
def unpersist(blocking: Boolean): this.type = {
sparkSession.sharedState.cacheManager.uncacheQuery(
sparkSession, logicalPlan, cascade = false, blocking)
this
}
Just pointing it out because this example from the docs don't compile since
unpersist() is not Unit
streamingDF.writeStream.foreachBatch { (batchDF: DataFrame, batchId: Long) =>
batchDF.persist()
batchDF.write.format(...).save(...) // location 1
batchDF.write.format(...).save(...) // location 2
batchDF.unpersist()}
Thanks!