Re: Null pointer exception with RDD while computing a method, creating dataframe.

2016-12-21 Thread Liang-Chi Hsieh
_code").parquet(.) - Liang-Chi Hsieh | @viirya Spark Technology Center http://www.spark.tc/ -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Null-pointer-exception-with-RDD-while-computing-a-method-creating-dataframe-tp20308p20328.html Sent from the Apa

Re: Null pointer exception with RDD while computing a method, creating dataframe.

2016-12-21 Thread satyajit vegesna
://www.spark.tc/ > -- > View this message in context: http://apache-spark- > developers-list.1001551.n3.nabble.com/Null-pointer- > exception-with-RDD-while-computing-a-method-creating- > dataframe-tp20308p20309.html > Sent from the Apache Spark Developers List mailing list archive at >

Re: Null pointer exception with RDD while computing a method, creating dataframe.

2016-12-20 Thread Liang-Chi Hsieh
Spark Technology Center http://www.spark.tc/ -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Null-pointer-exception-with-RDD-while-computing-a-method-creating-dataframe-tp20308p20309.html Sent from the Apache Spark Developers List mailing list archive at Nabbl

Null pointer exception with RDD while computing a method, creating dataframe.

2016-12-20 Thread satyajit vegesna
Hi All, PFB sample code , val df = spark.read.parquet() df.registerTempTable("df") val zip = df.select("zip_code").distinct().as[String].rdd def comp(zipcode:String):Unit={ val zipval = "SELECT * FROM df WHERE zip_code='$zipvalrepl'".replace("$zipvalrepl", zipcode) val data = spark.sql(zip