Hi All,IS NOT NULL is not working in programmatic sql. check below for input
output and code.
Input
====
10,IN
11,PK
12,US
13,UK
14,US
15,IN
16,
17,AS
18,AS
19,IR
20,As
val cntdat = sc.textFile("/user/poc_hortonworks/radha/gsd/sample.txt");
case class CNT (id:Int , code : String)
val cntdf = cntdat.map((f) => { val ff=f.split(",");new
CNT(ff(0).toInt,ff(1))}).toDF
cntdf.registerTempTable("cntids");
sqlContext.sql("SELECT MAX(id),code FROM cntids WHERE code is not null GROUP
BY code").show()
Output
=====
+---+----+
|_c0|code|
+---+----+
| 18| AS|
| 16| |
| 13| UK|
| 14| US|
| 20| As|
| 15| IN|
| 19| IR|
| 11| PK|
+---+----+
i am expecting the below one any idea, how to apply IS NOT NULL ?
+---+----+
|_c0|code|
+---+----+
| 18| AS|
| 13| UK|
| 14| US|
| 20| As|
| 15| IN|
| 19| IR|
| 11| PK|
+---+----+
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/IS-NOT-NULL-is-not-working-in-programmatic-SQL-in-spark-tp27317.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
