Yes that is the right issue. Variable substitution is not happening. I can't say much here as I haven't tried out this on 0.6 . The code on my blog post is based on 0.7 or higher version I guess.
Regards Bejoy KS Sent from handheld, please excuse typos. -----Original Message----- From: Techy Teck <comptechge...@gmail.com> Date: Sat, 4 Aug 2012 00:35:20 To: <user@hive.apache.org>; <bejoy...@yahoo.com> Reply-To: user@hive.apache.org Subject: Re: Passing date as command line arguments Hi Bejoy, I found something very strange. When I actually viewed the Hive log file, I found something like this- See the thing in red color. Why is it like so? *SessionStart SESSION_ID="rjamal_201208041433" TIME="1344065620956"* *QueryStart QUERY_STRING=" select * from lip_data_quality where dt = ' ${hiveconf:start_date}'" QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d" TIME="1344065629740"* *Counters plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":" select * from lip_data_quality where dt = '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}" TIME="1344065629743"* *QueryEnd QUERY_STRING=" select * from lip_data_quality where dt = '${hiveconf:start_date}'" QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d" QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="1344065629745"* *Counters plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":" select * from lip_data_quality where dt = '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}" TIME="1344065629745"* On Sat, Aug 4, 2012 at 12:30 AM, Techy Teck <comptechge...@gmail.com> wrote: > I cannot try that on Higher version of Hive as I am working in a company > where they have installed only Hive 0.6 on the Ares cluster. > > :( > > > > > On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <bejoy...@yahoo.com> wrote: > >> ** >> >> Try it on a higher version of hive and let me know if that doesn't work >> still. 0.9 should be good. >> >> >> Regards >> Bejoy KS >> >> Sent from handheld, please excuse typos. >> ------------------------------ >> *From: * Techy Teck <comptechge...@gmail.com> >> *Date: *Sat, 4 Aug 2012 00:26:44 -0700 >> *To: *<user@hive.apache.org>; <bejoy...@yahoo.com> >> *ReplyTo: * user@hive.apache.org >> *Subject: *Re: Passing date as command line arguments >> >> Yes it is String for me here. >> >> This is the table structure by the way- >> >> *create table lip_data_quality* >> *( buyer_id bigint,* >> * total_chkout bigint,* >> * total_errpds bigint* >> * )* >> * partitioned by (dt string)* >> *row format delimited fields terminated by '\t'* >> *stored as sequencefile* >> *location '/apps/hdmi-technology/b_apdpds/lip-data-quality'* >> *;* >> >> in which I am using dt in the where clause in my query. I am running *Hive >> 0.6.* >> >> >> >> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <bejoy...@yahoo.com> wrote: >> >>> ** >>> I tried the same query on my end, It is working fine for me without any >>> issues. >>> >>> By de way the data type for 'dt' is String itself right? >>> Regards >>> Bejoy KS >>> >>> Sent from handheld, please excuse typos. >>> ------------------------------ >>> *From: * Techy Teck <comptechge...@gmail.com> >>> *Date: *Sat, 4 Aug 2012 00:21:37 -0700 >>> *To: *<user@hive.apache.org>; <bejoy...@yahoo.com> >>> *ReplyTo: * user@hive.apache.org >>> *Subject: *Re: Passing date as command line arguments >>> >>> I was not able to understand. So what changes I need to make? This is my >>> below *hivetest1.hql-* >>> * >>> * >>> *select * from lip_data_quality where dt = '${hiveconf:start_date}';* >>> * >>> * >>> And by this I am executing the above hql file- >>> >>> *hive -f hivetest1.hql -hiveconf start_date=20120709* >>> * >>> * >>> * >>> * >>> Any thoughts what changes I need to make? >>> >>> >>> >>> >>> >>> >>> >>> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <bejoy...@yahoo.com> wrote: >>> >>>> ** >>>> Yes. >>>> >>>> From the logs the query being executed is >>>> select * from lip_data_quality where dt=20120709 >>>> >>>> But here the dt is not in quotes. It should be like >>>> select * from lip_data_quality where dt='20120709'; >>>> Regards >>>> Bejoy KS >>>> >>>> Sent from handheld, please excuse typos. >>>> ------------------------------ >>>> *From: * Techy Teck <comptechge...@gmail.com> >>>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700 >>>> *To: *<user@hive.apache.org>; Bejoy Ks<bejoy...@yahoo.com> >>>> *ReplyTo: * user@hive.apache.org >>>> *Subject: *Re: Passing date as command line arguments >>>> >>>> I found this thing from the Hive log. This is what you are looking for?- >>>> >>>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"* >>>> *QueryStart QUERY_STRING="select * from lip_data_quality where >>>> dt=20120709" >>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486" >>>> TIME="1344063597488"* >>>> *Counters >>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select >>>> * from lip_data_quality where dt=20120709"},"queryCo* >>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}" >>>> TIME="1344063597501"* >>>> *QueryEnd QUERY_STRING="select * from lip_data_quality where >>>> dt=20120709" >>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486" >>>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359* >>>> *7503"* >>>> *Counters >>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select >>>> * from lip_data_quality where dt=20120709"},"queryCo* >>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}" >>>> TIME="1344063597504"* >>>> * >>>> * >>>> * >>>> * >>>> >>>> >>>> >>>> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <comptechge...@gmail.com>wrote: >>>> >>>>> Yes that query gives me the result back correctly. >>>>> >>>>> How can I see the hive log Bejoy? Basically I am not sure which hive >>>>> log and what is the path where I can see that thing? >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <bejoy...@yahoo.com> wrote: >>>>> >>>>>> Hi Techy >>>>>> >>>>>> Are you getting the results for the same query directly from CLI, I >>>>>> mean does the below query return correct results >>>>>> select * from lip_data_quality where dt = '20120709'; >>>>>> >>>>>> If the above query returns results, can you check the hive logs and >>>>>> see what is the actual query being executed? >>>>>> >>>>>> Regards, >>>>>> Bejoy KS >>>>>> >>>>>> ------------------------------ >>>>>> *From:* Techy Teck <comptechge...@gmail.com> >>>>>> *To:* user@hive.apache.org >>>>>> *Cc:* Vijay Dirisala <tec...@gmail.com> >>>>>> *Sent:* Saturday, August 4, 2012 12:11 PM >>>>>> *Subject:* Re: Passing date as command line arguments >>>>>> >>>>>> Thanks Vijay for the suggestion. I also tried that and it still >>>>>> didn't worked for me. >>>>>> >>>>>> Any thoughts why it is not working? >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <tec...@gmail.com> wrote: >>>>>> >>>>>> There can't be a space in the variable name. Try this: >>>>>> ${hiveconf:start_date} >>>>>> >>>>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <comptechge...@gmail.com> >>>>>> wrote: >>>>>> > Can anyone help me out on this? >>>>>> > >>>>>> > I am stuck on this and I have no clue what wrong I am doing? >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <comptechge...@gmail.com> >>>>>> wrote: >>>>>> >> >>>>>> >> I have my below query in test1.hql file. I am trying to pass the >>>>>> date (dt) >>>>>> >> as the command line argument. >>>>>> >> >>>>>> >> >>>>>> >> select * from lip_data_quality where dt = '${hiveconf: >>>>>> start_date}'; >>>>>> >> >>>>>> >> >>>>>> >> So whenever I try to run the above test1.hql file like this- >>>>>> >> >>>>>> >> hive -f hivetest1.hql -hiveconf start_date=20120709 >>>>>> >> >>>>>> >> I get zero records back. But the data is there in that table for >>>>>> that >>>>>> >> date. Why is it so? Something wrong I am doing? >>>>>> >> >>>>>> >> Can anyone help me out here? I was following Bejoy's Article here >>>>>> >> >>>>>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html >>>>>> >> >>>>>> >> >>>>>> > >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >