Re: small files with hive and hadoop

2011-01-31 Thread Ajo Fod
I've noticed that it takes a while for each map job to be set up in hive ... and the way I set up the job I noticed that there were as many maps as files/buckets. I read a recommendation somewhere to design jobs such that they take at least a minute. Cheers, -Ajo. On Mon, Jan 31, 2011 at 8:08 AM

Re: small files with hive and hadoop

2011-01-31 Thread Edward Capriolo
On Mon, Jan 31, 2011 at 11:08 AM, wrote: > Hello, > > I like to do a reporting with Hive on something like tracking data. > The raw data which is about 2 gigs or more a day I want to query with hive. > This works already for me, no problem. > Also I want to cascade down the reporting data to som

small files with hive and hadoop

2011-01-31 Thread hive1
Hello, I like to do a reporting with Hive on something like tracking data. The raw data which is about 2 gigs or more a day I want to query with hive. This works already for me, no problem. Also I want to cascade down the reporting data to something like client, date, something in Hive like part