Re: FW: Big table join optimization

2014-01-30 Thread Stephen Sprague
lets see the minimal query that shows your problem with some comments about cardinality of the tables in the join. maybe there could be a crude workaround using a temp table or some such device if nothing jumps out at us. On Thu, Jan 30, 2014 at 4:07 PM, Guy Doulberg wrote: > > hi guys > > I a

FW: Big table join optimization

2014-01-30 Thread Guy Doulberg
hi guys I am trying to optimize a hive join query, I have a join of two big tables. The join between them is taking too long, no matter how many reducers I set, there are always two reducers struggling to finish in the end of the job The job not always ends, sometime it fails with memory prob