Hi, Could anyone please provide me your suggestions on how to resolve the issue that I am facing with not contains code on dataframe column.
Here is the code. My dataframe is not getting filtered with below conditions. I even tried not and ! on Column. any suggestions? def filterDatapointRawCountsDF(vin: Column): Column = { import org.apache.spark.sql.functions.not val filterColumn: Column = { not(vin.contains("VIN")) || not(vin.contains("Ÿ")) || not(vin.contains("0123456789ABCDEFG")) } filterColumn; } }