Variable support was not added until hive 0.7.0
On Sat, Aug 4, 2012 at 3:58 AM, Lefty Leverenz <le...@hortonworks.com> wrote:
> Does Hive 0.6 support underscores in variable names?
>
> Just a thought. The examples I've seen contain letters and dots (".") but
> no other characters.
>
> – Lefty
>
>
>
>
> On Sat, Aug 4, 2012 at 12:39 AM, Bejoy KS <bejoy...@yahoo.com> wrote:
>>
>> Yes that is the right issue. Variable substitution is not happening. I
>> can't say much here as I haven't tried out this on 0.6 . The code on my blog
>> post is based on 0.7 or higher version I guess.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ________________________________
>> From: Techy Teck <comptechge...@gmail.com>
>> Date: Sat, 4 Aug 2012 00:35:20 -0700
>> To: <user@hive.apache.org>; <bejoy...@yahoo.com>
>> ReplyTo: user@hive.apache.org
>> Subject: Re: Passing date as command line arguments
>>
>> Hi Bejoy,
>>
>> I found something very strange. When I actually viewed the Hive log file,
>> I found something like this- See the thing in red color. Why is it like so?
>>
>> SessionStart SESSION_ID="rjamal_201208041433" TIME="1344065620956"
>> QueryStart QUERY_STRING=" select * from lip_data_quality where dt =
>> '${hiveconf:start_date}'"
>> QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
>> TIME="1344065629740"
>> Counters
>> plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
>> select * from lip_data_quality where dt =
>> '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>> TIME="1344065629743"
>> QueryEnd QUERY_STRING=" select * from lip_data_quality where dt =
>> '${hiveconf:start_date}'"
>> QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="1344065629745"
>> Counters
>> plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
>> select * from lip_data_quality where dt =
>> '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>> TIME="1344065629745"
>>
>> On Sat, Aug 4, 2012 at 12:30 AM, Techy Teck <comptechge...@gmail.com>
>> wrote:
>>>
>>> I cannot try that on Higher version of Hive as I am working in a company
>>> where they have installed only Hive 0.6 on the Ares cluster.
>>>
>>> :(
>>>
>>>
>>>
>>>
>>> On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <bejoy...@yahoo.com> wrote:
>>>>
>>>>
>>>> Try it on a higher version of hive and let me know if that doesn't work
>>>> still. 0.9 should be good.
>>>>
>>>>
>>>> Regards
>>>> Bejoy KS
>>>>
>>>> Sent from handheld, please excuse typos.
>>>> ________________________________
>>>> From: Techy Teck <comptechge...@gmail.com>
>>>> Date: Sat, 4 Aug 2012 00:26:44 -0700
>>>> To: <user@hive.apache.org>; <bejoy...@yahoo.com>
>>>> ReplyTo: user@hive.apache.org
>>>> Subject: Re: Passing date as command line arguments
>>>>
>>>> Yes it is String for me here.
>>>>
>>>> This is the table structure by the way-
>>>>
>>>> create table lip_data_quality
>>>> ( buyer_id bigint,
>>>> total_chkout bigint,
>>>> total_errpds bigint
>>>> )
>>>> partitioned by (dt string)
>>>> row format delimited fields terminated by '\t'
>>>> stored as sequencefile
>>>> location '/apps/hdmi-technology/b_apdpds/lip-data-quality'
>>>> ;
>>>>
>>>> in which I am using dt in the where clause in my query. I am running
>>>> Hive 0.6.
>>>>
>>>>
>>>>
>>>> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <bejoy...@yahoo.com> wrote:
>>>>>
>>>>> I tried the same query on my end, It is working fine for me without any
>>>>> issues.
>>>>>
>>>>> By de way the data type for 'dt' is String itself right?
>>>>> Regards
>>>>> Bejoy KS
>>>>>
>>>>> Sent from handheld, please excuse typos.
>>>>> ________________________________
>>>>> From: Techy Teck <comptechge...@gmail.com>
>>>>> Date: Sat, 4 Aug 2012 00:21:37 -0700
>>>>> To: <user@hive.apache.org>; <bejoy...@yahoo.com>
>>>>> ReplyTo: user@hive.apache.org
>>>>> Subject: Re: Passing date as command line arguments
>>>>>
>>>>> I was not able to understand. So what changes I need to make? This is
>>>>> my below hivetest1.hql-
>>>>>
>>>>> select * from lip_data_quality where dt = '${hiveconf:start_date}';
>>>>>
>>>>> And by this I am executing the above hql file-
>>>>>
>>>>> hive -f hivetest1.hql -hiveconf start_date=20120709
>>>>>
>>>>>
>>>>> Any thoughts what changes I need to make?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <bejoy...@yahoo.com> wrote:
>>>>>>
>>>>>> Yes.
>>>>>>
>>>>>> From the logs the query being executed is
>>>>>> select * from lip_data_quality where dt=20120709
>>>>>>
>>>>>> But here the dt is not in quotes. It should be like
>>>>>> select * from lip_data_quality where dt='20120709';
>>>>>> Regards
>>>>>> Bejoy KS
>>>>>>
>>>>>> Sent from handheld, please excuse typos.
>>>>>> ________________________________
>>>>>> From: Techy Teck <comptechge...@gmail.com>
>>>>>> Date: Sat, 4 Aug 2012 00:07:07 -0700
>>>>>> To: <user@hive.apache.org>; Bejoy Ks<bejoy...@yahoo.com>
>>>>>> ReplyTo: user@hive.apache.org
>>>>>> Subject: Re: Passing date as command line arguments
>>>>>>
>>>>>> I found this thing from the Hive log. This is what you are looking
>>>>>> for?-
>>>>>>
>>>>>> SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"
>>>>>> QueryStart QUERY_STRING="select * from lip_data_quality where
>>>>>> dt=20120709"
>>>>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>>>>> TIME="1344063597488"
>>>>>> Counters
>>>>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>>>>> * from lip_data_quality where dt=20120709"},"queryCo
>>>>>>
>>>>>> unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>>>>>> TIME="1344063597501"
>>>>>> QueryEnd QUERY_STRING="select * from lip_data_quality where
>>>>>> dt=20120709"
>>>>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>>>>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359
>>>>>> 7503"
>>>>>> Counters
>>>>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>>>>> * from lip_data_quality where dt=20120709"},"queryCo
>>>>>>
>>>>>> unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>>>>>> TIME="1344063597504"
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <comptechge...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Yes that query gives me the result back correctly.
>>>>>>>
>>>>>>> How can I see the hive log Bejoy? Basically I am not sure which hive
>>>>>>> log and what is the path where I can see that thing?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <bejoy...@yahoo.com> wrote:
>>>>>>>>
>>>>>>>> Hi Techy
>>>>>>>>
>>>>>>>> Are you getting the results for the same query directly from CLI, I
>>>>>>>> mean does the below query return correct results
>>>>>>>> select * from lip_data_quality where dt = '20120709';
>>>>>>>>
>>>>>>>> If the above query returns results, can you check the hive logs and
>>>>>>>> see what is the actual query being executed?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Bejoy KS
>>>>>>>>
>>>>>>>> ________________________________
>>>>>>>> From: Techy Teck <comptechge...@gmail.com>
>>>>>>>> To: user@hive.apache.org
>>>>>>>> Cc: Vijay Dirisala <tec...@gmail.com>
>>>>>>>> Sent: Saturday, August 4, 2012 12:11 PM
>>>>>>>> Subject: Re: Passing date as command line arguments
>>>>>>>>
>>>>>>>> Thanks Vijay for the suggestion. I also tried that and it still
>>>>>>>> didn't worked for me.
>>>>>>>>
>>>>>>>> Any thoughts why it is not working?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <tec...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> There can't be a space in the variable name. Try this:
>>>>>>>> ${hiveconf:start_date}
>>>>>>>>
>>>>>>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck
>>>>>>>> <comptechge...@gmail.com> wrote:
>>>>>>>> > Can anyone help me out on this?
>>>>>>>> >
>>>>>>>> > I am stuck on this and I have no clue what wrong I am doing?
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck
>>>>>>>> > <comptechge...@gmail.com> wrote:
>>>>>>>> >>
>>>>>>>> >> I have my below query in test1.hql file. I am trying to pass the
>>>>>>>> >> date (dt)
>>>>>>>> >> as the command line argument.
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> >> select * from lip_data_quality where dt = '${hiveconf:
>>>>>>>> >> start_date}';
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> >> So whenever I try to run the above test1.hql file like this-
>>>>>>>> >>
>>>>>>>> >> hive -f hivetest1.hql -hiveconf start_date=20120709
>>>>>>>> >>
>>>>>>>> >> I get zero records back. But the data is there in that table for
>>>>>>>> >> that
>>>>>>>> >> date. Why is it so? Something wrong I am doing?
>>>>>>>> >>
>>>>>>>> >> Can anyone help me out here? I was following Bejoy's Article here
>>>>>>>> >>
>>>>>>>> >> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> >
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
