Hi,
Yes, this is what I'm doing. I'm using hiveContext.hql() to run my
query.
But, the problem still happens.
On Tue, Jan 20, 2015 at 7:24 PM, DEVAN M.S. <[email protected]> wrote:
> Add one more library
>
> libraryDependencies += "org.apache.spark" % "spark-hive_2.10" % "1.2.0"
>
>
> val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc)
>
> repalce sqlContext with hiveContext. Its working while using HiveContext
> for me.
>
>
>
> Devan M.S. | Research Associate | Cyber Security | AMRITA VISHWA
> VIDYAPEETHAM | Amritapuri | Cell +919946535290 |
>
>
> On Tue, Jan 20, 2015 at 4:45 PM, DEVAN M.S. <[email protected]> wrote:
>
>> Which context are you using HiveContext or SQLContext ? Can you try with
>> HiveContext
>> ??
>>
>>
>> Devan M.S. | Research Associate | Cyber Security | AMRITA VISHWA
>> VIDYAPEETHAM | Amritapuri | Cell +919946535290 |
>>
>>
>> On Tue, Jan 20, 2015 at 3:49 PM, Xuelin Cao <[email protected]>
>> wrote:
>>
>>>
>>> Hi, I'm using Spark 1.2
>>>
>>>
>>> On Tue, Jan 20, 2015 at 5:59 PM, Wang, Daoyuan <[email protected]>
>>> wrote:
>>>
>>>> Hi Xuelin,
>>>>
>>>>
>>>>
>>>> What version of Spark are you using?
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Daoyuan
>>>>
>>>>
>>>>
>>>> *From:* Xuelin Cao [mailto:[email protected]]
>>>> *Sent:* Tuesday, January 20, 2015 5:22 PM
>>>> *To:* User
>>>> *Subject:* IF statement doesn't work in Spark-SQL?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Hi,
>>>>
>>>>
>>>>
>>>> I'm trying to migrate some hive scripts to Spark-SQL. However, I
>>>> found some statement is incompatible in Spark-sql.
>>>>
>>>>
>>>>
>>>> Here is my SQL. And the same SQL works fine in HIVE environment.
>>>>
>>>>
>>>>
>>>> SELECT
>>>>
>>>> *if(ad_user_id>1000, 1000, ad_user_id) as user_id*
>>>>
>>>> FROM
>>>>
>>>> ad_search_keywords
>>>>
>>>>
>>>>
>>>> What I found is, the parser reports error on the "*if*" statement:
>>>>
>>>>
>>>>
>>>> No function to evaluate expression. type: AttributeReference, tree:
>>>> ad_user_id#4
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Anyone have any idea about this?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>
