Re: Using Hive generated SeqeunceFiles and RC files with Java MapReduce and PIG

2014-01-28 Thread Thilina Gunarathne
Thanks for the information Edward. When you use the default Serde (lazySerde) and sequence files hive writes a > SequenceFile(create table x stored as sequence file) , the key is null > and hive serializes all the columns into a Text Writable that is easy for > other tools to read. > Does thi

Re: Using Hive generated SeqeunceFiles and RC files with Java MapReduce and PIG

2014-01-28 Thread Edward Capriolo
When you use the default Serde (lazySerde) and sequence files hive writes a SequenceFile(create table x stored as sequence file) , the key is null and hive serializes all the columns into a Text Writable that is easy for other tools to read. Hive does not dictate the input format or the output

Using Hive generated SeqeunceFiles and RC files with Java MapReduce and PIG

2014-01-28 Thread Thilina Gunarathne
Hi, We have a requirement to store a large data set (more than 5TB) mapped to a Hive table. This Hive table would be populated (and appended periodically) using a Hive query from another Hive table. In addition to the Hive queries, we need to be able to run Java MapReduce and preferably Pig jobs as