I want to apply null comparison to a column in sqlcontext.sql, is there any
way to achieve this?
On Jul 10, 2016 8:55 PM, "Radha krishna" <grkmc...@gmail.com> wrote:
> Ok thank you, how to achieve the requirement.
>
> On Sun, Jul 10, 2016 at 8:44 PM, Sean Owen <so...@cloudera.com> wrote:
>
>> It doesn't look like you have a NULL field, You have a string-value
>> field with an empty string.
>>
>> On Sun, Jul 10, 2016 at 3:19 PM, Radha krishna <grkmc...@gmail.com>
>> wrote:
>> > Hi All,IS NOT NULL is not working in programmatic sql. check below for
>> input
>> > output and code.
>> >
>> > Input
>> > ====
>> > 10,IN
>> > 11,PK
>> > 12,US
>> > 13,UK
>> > 14,US
>> > 15,IN
>> > 16,
>> > 17,AS
>> > 18,AS
>> > 19,IR
>> > 20,As
>> >
>> > val cntdat = sc.textFile("/user/poc_hortonworks/radha/gsd/sample.txt");
>> > case class CNT (id:Int , code : String)
>> > val cntdf = cntdat.map((f) => { val ff=f.split(",");new
>> > CNT(ff(0).toInt,ff(1))}).toDF
>> > cntdf.registerTempTable("cntids");
>> > sqlContext.sql("SELECT MAX(id),code FROM cntids WHERE code is not null
>> GROUP
>> > BY code").show()
>> >
>> > Output
>> > =====
>> > +---+----+
>> > |_c0|code|
>> > +---+----+
>> > | 18| AS|
>> > | 16| |
>> > | 13| UK|
>> > | 14| US|
>> > | 20| As|
>> > | 15| IN|
>> > | 19| IR|
>> > | 11| PK|
>> > +---+----+
>> >
>> > i am expecting the below one any idea, how to apply IS NOT NULL ?
>> >
>> > +---+----+
>> > |_c0|code|
>> > +---+----+
>> > | 18| AS|
>> > | 13| UK|
>> > | 14| US|
>> > | 20| As|
>> > | 15| IN|
>> > | 19| IR|
>> > | 11| PK|
>> > +---+----+
>> >
>> >
>> >
>> > Thanks & Regards
>> > Radha krishna
>> >
>> >
>>
>
>
>
> --
>
>
>
>
>
>
>
>
> Thanks & Regards
> Radha krishna
>
>
>
