Re: Delete NA in a dataframe

2015-08-04 Thread clark djilo kuissu
Thank you Peter I try this Le Mardi 4 août 2015 15h02, Peter Rudenko a écrit : Hi Clark, the problem is that in this dataset null values represented as NA marker. Spark-csv doesn't have configurable null values marker (i've made a PR with it some time ago: https://github.com/da

Re: Delete NA in a dataframe

2015-08-04 Thread Peter Rudenko
Hi Clark, the problem is that in this dataset null values represented as NA marker. Spark-csv doesn't have configurable null values marker (i've made a PR with it some time ago: https://github.com/databricks/spark-csv/pull/76). So one option for you is to do post filtering, something like thi