Re: IS NOT NULL is not working in programmatic SQL in spark

2016-07-10 Thread Takeshi Yamamuro
Hi, One of solutions to use `spark-csv` (See: https://github.com/databricks/spark-csv#features). To load NULL, you can use `nullValue` there. // maropu On Mon, Jul 11, 2016 at 1:14 AM, Radha krishna wrote: > I want to apply null comparison to a column in sqlcontext.sql, is there > any way to

Re: IS NOT NULL is not working in programmatic SQL in spark

2016-07-10 Thread Radha krishna
I want to apply null comparison to a column in sqlcontext.sql, is there any way to achieve this? On Jul 10, 2016 8:55 PM, "Radha krishna" wrote: > Ok thank you, how to achieve the requirement. > > On Sun, Jul 10, 2016 at 8:44 PM, Sean Owen wrote: > >> It doesn't look like you have a NULL field,

Re: IS NOT NULL is not working in programmatic SQL in spark

2016-07-10 Thread Radha krishna
Ok thank you, how to achieve the requirement. On Sun, Jul 10, 2016 at 8:44 PM, Sean Owen wrote: > It doesn't look like you have a NULL field, You have a string-value > field with an empty string. > > On Sun, Jul 10, 2016 at 3:19 PM, Radha krishna wrote: > > Hi All,IS NOT NULL is not working in

Re: IS NOT NULL is not working in programmatic SQL in spark

2016-07-10 Thread Sean Owen
It doesn't look like you have a NULL field, You have a string-value field with an empty string. On Sun, Jul 10, 2016 at 3:19 PM, Radha krishna wrote: > Hi All,IS NOT NULL is not working in programmatic sql. check below for input > output and code. > > Input > > 10,IN > 11,PK > 12,US > 13,UK