Map Red SequenceFile output to Hive table

2012-01-26 Thread rk vishu
Hello All, I have a mapred job that does transfermation and outputs to a compresses SequenceFile (by using org.apache.hadoop.mapred.SequenceFileOutputFormat) I am able to attach the output to a external hive table (stored as sequncefile). When i query it ignores the first column value from the fi

Re: better partitioning strategy in hive

2012-02-18 Thread rk vishu
> Hello All, > > We have a hive table partitioned by date and hour(330 columns). We have 5 > years worth of data for the table. Each hourly partition have around 800MB. > So total 43,800 partitions with one file per partition. > > When we run select count(*) from table, hive is taking for ever to s