Which context are you using HiveContext or SQLContext ? Can you try with HiveContext ??
Devan M.S. | Research Associate | Cyber Security | AMRITA VISHWA VIDYAPEETHAM | Amritapuri | Cell +919946535290 | On Tue, Jan 20, 2015 at 3:49 PM, Xuelin Cao <xuelincao2...@gmail.com> wrote: > > Hi, I'm using Spark 1.2 > > > On Tue, Jan 20, 2015 at 5:59 PM, Wang, Daoyuan <daoyuan.w...@intel.com> > wrote: > >> Hi Xuelin, >> >> >> >> What version of Spark are you using? >> >> >> >> Thanks, >> >> Daoyuan >> >> >> >> *From:* Xuelin Cao [mailto:xuelincao2...@gmail.com] >> *Sent:* Tuesday, January 20, 2015 5:22 PM >> *To:* User >> *Subject:* IF statement doesn't work in Spark-SQL? >> >> >> >> >> >> Hi, >> >> >> >> I'm trying to migrate some hive scripts to Spark-SQL. However, I >> found some statement is incompatible in Spark-sql. >> >> >> >> Here is my SQL. And the same SQL works fine in HIVE environment. >> >> >> >> SELECT >> >> *if(ad_user_id>1000, 1000, ad_user_id) as user_id* >> >> FROM >> >> ad_search_keywords >> >> >> >> What I found is, the parser reports error on the "*if*" statement: >> >> >> >> No function to evaluate expression. type: AttributeReference, tree: >> ad_user_id#4 >> >> >> >> >> >> Anyone have any idea about this? >> >> >> >> >> > >
