Re: Record too large for Tez in-memory buffer...

2016-02-11 Thread Gautam
setting tez.task.scale.memory.enabled=false; failed the job sooner. But the fix from TEZ-2575 worked! Upgraded from 0.7.0 to 0.7.1 and applied https://github.com/apache/tez/commit/0e155e7185d1350f64dead488103777295ac76d1 Goes through without any fatal issues. Will continue testing / benchmarki

Re: Record too large for Tez in-memory buffer...

2016-02-10 Thread Gautam
Thanks Gopal! I'l look at options provided. On Wed, Feb 10, 2016 at 7:46 PM, Gautam wrote: > Here's the json version. > > On Wed, Feb 10, 2016 at 7:44 PM, Gautam wrote: > >> Whoops.. meant to send the tez explain earlier. Here's the Tez query >> plan. Good to know there's a fix .. Is there a ji

Re: Record too large for Tez in-memory buffer...

2016-02-10 Thread Gautam
Here's the json version. On Wed, Feb 10, 2016 at 7:44 PM, Gautam wrote: > Whoops.. meant to send the tez explain earlier. Here's the Tez query plan. > Good to know there's a fix .. Is there a jira that talks about this issue? Coz > I couldn't find one. Maybe I can alter the query a bit to filter

Re: Record too large for Tez in-memory buffer...

2016-02-10 Thread Gautam
Whoops.. meant to send the tez explain earlier. Here's the Tez query plan. Good to know there's a fix .. Is there a jira that talks about this issue? Coz I couldn't find one. Maybe I can alter the query a bit to filter these out. Cheers, -Gautam. On Wed, Feb 10, 2016 at 7:32 PM, Gopal Vijayaragh

Re: Record too large for Tez in-memory buffer...

2016-02-10 Thread Gopal Vijayaraghavan
Hey, > Trying to benchmark with Hive on Tez causes the following error. >Admittedly these are some very large looking records .. the same job runs >fine on MR2. ... > I'v attached the query explain tree. It fails in the very last reducer >phase .. Can you attach the explain plan with hive.exec