Re: Merging different HDFS file for HIVE

2013-07-26 Thread Sanjay Subramanian
...@wizecommerce.com>> Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Date: Friday, July 26, 2013 6:23 PM To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Subject: Re: Mergi

Re: Merging different HDFS file for HIVE

2013-07-26 Thread Sanjay Subramanian
m: Stephen Sprague mailto:sprag...@gmail.com>> Reply-To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Date: Friday, July 26, 2013 4:37 PM To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.o

Re: Merging different HDFS file for HIVE

2013-07-26 Thread Stephen Sprague
i like #2. so you have three, say, external tables representing your three feed files. After the third and final file is loaded then join 'em all together - maybe make the table partitioned for one per day. for example: alter table final add partition (datekey=MMDD); insert overwrite table f

Re: Merging different HDFS file for HIVE

2013-07-26 Thread Nitin Pawar
Option 1 ) Use pig or oozie, write a workflow and join the files to a single file Option 2 ) Create a temp table for each of the different file and then join them to a single table and delete temp table Option 3 ) don't do anything, change your queries to look at three different files when they que

Merging different HDFS file for HIVE

2013-07-26 Thread Ramasubramanian Narayanan
Hi, Please help in providing solution for the below problem... this scenario is applicable in Banking atleast... I have a HIVE table with the below structure... Hive Table: Field1 ... Field 10 For the above table, I will get the values for each feed in different file. You can imagine that thes