Re: Record too large for Tez in-memory buffer...

2016-02-11 Thread Gautam
setting tez.task.scale.memory.enabled=false; failed the job sooner. But the fix from TEZ-2575 worked! Upgraded from 0.7.0 to 0.7.1 and applied https://github.com/apache/tez/commit/0e155e7185d1350f64dead488103777295ac76d1 Goes through without any fatal issues. Will continue testing / benchmarki

Re: Record too large for Tez in-memory buffer...

2016-02-10 Thread Gautam
Thanks Gopal! I'l look at options provided. On Wed, Feb 10, 2016 at 7:46 PM, Gautam wrote: > Here's the json version. > > On Wed, Feb 10, 2016 at 7:44 PM, Gautam wrote: > >> Whoops.. meant to send the tez explain earlier. Here's the Tez query >> plan. Good to know there's a fix .. Is there a ji

Re: Record too large for Tez in-memory buffer...

2016-02-10 Thread Gautam
Here's the json version. On Wed, Feb 10, 2016 at 7:44 PM, Gautam wrote: > Whoops.. meant to send the tez explain earlier. Here's the Tez query plan. > Good to know there's a fix .. Is there a jira that talks about this issue? Coz > I couldn't find one. Maybe I can alter the query a bit to filter

Re: Record too large for Tez in-memory buffer...

2016-02-10 Thread Gautam
Whoops.. meant to send the tez explain earlier. Here's the Tez query plan. Good to know there's a fix .. Is there a jira that talks about this issue? Coz I couldn't find one. Maybe I can alter the query a bit to filter these out. Cheers, -Gautam. On Wed, Feb 10, 2016 at 7:32 PM, Gopal Vijayaragh

Re: Record too large for Tez in-memory buffer...

2016-02-10 Thread Gopal Vijayaraghavan
Hey, > Trying to benchmark with Hive on Tez causes the following error. >Admittedly these are some very large looking records .. the same job runs >fine on MR2. ... > I'v attached the query explain tree. It fails in the very last reducer >phase .. Can you attach the explain plan with hive.exec

Record too large for Tez in-memory buffer...

2016-02-10 Thread Gautam
Hello , Trying to benchmark with Hive on Tez causes the following error. Admittedly these are some very large looking records .. the same job runs fine on MR2. I'v attached the query explain tree. It fails in the very last reducer phase .. *Execution:* -