RE: File Path and Partition names

2012-10-02 Thread carla.staeben
Yep, dynamic. Let me know if you figure something out. I'd hate to have to go through all of the trouble to etl the data and then create tables on top with the alter table command. Such a waste of time and effort... Carla -Original Message- From: ext Doug Houck [mailto:doug.ho...@tru

RE: File Path and Partition names

2012-10-02 Thread carla.staeben
Thanks Bejoy, I was kind of hoping to avoid all of the 'extra' work...it would be nice if hive didn't include the partition name in the path creation...I was hoping that there was a 'set' parameter/config I was missing. Thanks Carla From: ext Bejoy KS [mailto:bejoy...@yahoo.com] Sent: Tuesday,

File Path and Partition names

2012-10-02 Thread carla.staeben
Quick question about using hive to create new hdfs file paths. Generally speaking, we like to keep our data files with a path similar to Dataset/year/month/day/hour I need to create a new table in hive and populate it with data from a different dataset, using a HiveQL query. If I do this: CREA

RE: How to set default value for a certain field?

2012-09-05 Thread carla.staeben
Well, you could create the table and then insert overwrite from the file: Insert overwrite Select id, case when value = '' then 3 else value end as value >From testtest; Carla From: ext Philip Tromans [mailto:philip.j.trom...@gmail.com] Sent: Wednesday, September 05, 2012 07:52 To: user@hive.apa

RE: (Get the current date -1) in Hive

2012-08-06 Thread carla.staeben
In the case here it literally is taking the UNIX timestamp, formatting it in -mm-dd format and then subtracting the specified integer (in this case 1) Sent from my Lumia 900 From: ext Techy Teck Sent: 8/6/2012 3:37 PM To: user@hive.apache.org Subject: Re: (Get

RE: (Get the current date -1) in Hive

2012-08-06 Thread carla.staeben
If you are just using it in a query, you can do this: date_sub(FROM_UNIXTIME(UNIX_TIMESTAMP(),'-MM-dd') , 1) I generally do my date calculations in a shell script and pass them in with a hiveconf variable. Carla -Original Message- From: ext Yue Guan [mailto:pipeha...@gmail.com] Sen

RE: Variable Substitution Depth Limit

2012-03-30 Thread carla.staeben
Sorry, hit send too fast...As a work around for now, what I ended up doing was using the 'sed' command to replace all of my variables with a shell script. Kind of a hack, but it worked in the end for what I needed. Carla -Original Message- From: Staeben Carla (Nokia-LC/Boston) Sent: F

RE: Variable Substitution Depth Limit

2012-03-30 Thread carla.staeben
Yeah, you're not the only one who's run into that issue. There is an open Jira item for it, so they're aware we'd like it configurable anyway... https://issues.apache.org/jira/browse/HIVE-2021 -Original Message- From: ext Edward Capriolo [mailto:edlinuxg...@gmail.com] Sent: Friday, Ma

RE: Hive CLI and Standalone Server : Need Suggestion

2012-03-19 Thread carla.staeben
Great topic as I was wondering a similar thing this morning...I want to use oozie to execute my hive job, but I have to pass the job parameters that I generate with a shell script. Some of the literature that I've seen says that oozie may or may not allow for calling shell scripts. Is that tru