Can you share your code ?
Devan M.S. | Research Associate | Cyber Security | AMRITA VISHWA VIDYAPEETHAM | Amritapuri | Cell +919946535290 | On Tue, Jan 20, 2015 at 5:03 PM, Xuelin Cao <[email protected]> wrote: > > Hi, > > Yes, this is what I'm doing. I'm using hiveContext.hql() to run my > query. > > But, the problem still happens. > > > > On Tue, Jan 20, 2015 at 7:24 PM, DEVAN M.S. <[email protected]> wrote: > >> Add one more library >> >> libraryDependencies += "org.apache.spark" % "spark-hive_2.10" % "1.2.0" >> >> >> val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc) >> >> repalce sqlContext with hiveContext. Its working while using HiveContext >> for me. >> >> >> >> Devan M.S. | Research Associate | Cyber Security | AMRITA VISHWA >> VIDYAPEETHAM | Amritapuri | Cell +919946535290 | >> >> >> On Tue, Jan 20, 2015 at 4:45 PM, DEVAN M.S. <[email protected]> wrote: >> >>> Which context are you using HiveContext or SQLContext ? Can you try >>> with HiveContext ?? >>> >>> >>> Devan M.S. | Research Associate | Cyber Security | AMRITA VISHWA >>> VIDYAPEETHAM | Amritapuri | Cell +919946535290 | >>> >>> >>> On Tue, Jan 20, 2015 at 3:49 PM, Xuelin Cao <[email protected]> >>> wrote: >>> >>>> >>>> Hi, I'm using Spark 1.2 >>>> >>>> >>>> On Tue, Jan 20, 2015 at 5:59 PM, Wang, Daoyuan <[email protected]> >>>> wrote: >>>> >>>>> Hi Xuelin, >>>>> >>>>> >>>>> >>>>> What version of Spark are you using? >>>>> >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Daoyuan >>>>> >>>>> >>>>> >>>>> *From:* Xuelin Cao [mailto:[email protected]] >>>>> *Sent:* Tuesday, January 20, 2015 5:22 PM >>>>> *To:* User >>>>> *Subject:* IF statement doesn't work in Spark-SQL? >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Hi, >>>>> >>>>> >>>>> >>>>> I'm trying to migrate some hive scripts to Spark-SQL. However, I >>>>> found some statement is incompatible in Spark-sql. >>>>> >>>>> >>>>> >>>>> Here is my SQL. And the same SQL works fine in HIVE environment. >>>>> >>>>> >>>>> >>>>> SELECT >>>>> >>>>> *if(ad_user_id>1000, 1000, ad_user_id) as user_id* >>>>> >>>>> FROM >>>>> >>>>> ad_search_keywords >>>>> >>>>> >>>>> >>>>> What I found is, the parser reports error on the "*if*" >>>>> statement: >>>>> >>>>> >>>>> >>>>> No function to evaluate expression. type: AttributeReference, tree: >>>>> ad_user_id#4 >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Anyone have any idea about this? >>>>> >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>> >> >
