RE: how to let one map task read multiple files?

2011-08-27 Thread Aggarwal, Vaibhav
CombineFileInputFormat can be used to combine multiple files into one map task. But CombineFileInputFormat does not attempt to combine compressed files. It defaults to the HiveFileInputFormat which creates at least one map task per file. 7G of data is not a lot for 3 node cluster to process and y

how to let one map task read multiple files?

2011-08-27 Thread Daniel,Wu
I have a files of 7G, and the load using the command of load data local inpath '/home/oracle/store_sales.csv' into table store_sales; That file is not compressed, so I want to compress the table to make it work faster ( I don't know how to let hive work on a compress file directly), So I use t