Spark SQL has had an na.fill function on it since at least 2.1. Would that work for you?
https://spark.apache.org/docs/2.1.0/api/java/org/apache/spark/sql/DataFrameNaFunctions.html On Mon, Apr 29, 2019 at 4:57 PM Shixiong(Ryan) Zhu <shixi...@databricks.com> wrote: > Hey Snehasish, > > Do you have a reproducer for this issue? > > Best Regards, > Ryan > > > On Wed, Apr 24, 2019 at 7:24 AM SNEHASISH DUTTA <info.snehas...@gmail.com> > wrote: > >> Hi, >> >> While writing to kafka using spark structured streaming , if all the >> values in certain column are Null it gets dropped >> Is there any way to override this , other than using na.fill functions >> >> Regards, >> Snehasish >> > -- Thanks, Jason
