Hi
Before dealing the issue itself, can you please fix the query?
There are 3 aliased tables - Table1 (A), Table2 (B) & Table2 (mb) but you’re
using additional 2 aliases – ma & adi1.
Thanks
Dudu
select A.Col1,A.Col2B.Col3
From Table1 A
LEFT OUTER JOIN Table2 B
ON A.Col3= B.Col3
AND
Yes, I've reverted to the default there. Thanks!
On Mon, May 2, 2016 at 7:47 PM, Prasanth Jayachandran <
pjayachand...@hortonworks.com> wrote:
> Hi Matt
>
> As Gopal mentioned below you might have to unset
> hive.exec.orc.memory.pool=1.0;
> since some memory is required for sorting. Try running
Hi Matt
As Gopal mentioned below you might have to unset hive.exec.orc.memory.pool=1.0;
since some memory is required for sorting. Try running with the defaults for
hive.exec.orc.memory.pool.
Thanks
Prasanth
On May 2, 2016, at 9:41 PM, Matt Olson
mailto:maolso...@gmail.com>> wrote:
Hi Prasant
Hi Prasanth,
Thank you for the helpful information. I have been using the default ORC
stripe size, which I believe is 67,108,864 bytes.
I was able to remove the constant value for dt as you suggested, and set
hive.optimize.sort.dynamic.partition=true. I saw in the new explain plan
that the partit
Hi,
Is there a way to implement not exists in Hive. I'm using Hive 1.2. I'm
getting below error
"Unsupported SubQuery Expression '1': Only SubQuery expressions that are
top level conjuncts are allowed"
_Query:_
_
_
select A.Col1,A.Col2B.Col3
From Table1 A
LEFT OUTER JOIN Table2 B
Hi Matt
So it looks like you are hitting the issue that I had mentioned previously.
You might need to apply the patch from HIVE-12893. Alternatively, if dt has
only one possible value then
its better to remove the constant value for dt and the where condition. This
will enable sorted dynamic par
Hi Prasanth,
Here is the explain plan for the insert query:
OK
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-7 depends on stages: Stage-1 , consists of Stage-4, Stage-3, Stage-5
Stage-4
Stage-0 depends on stages: Stage-4, Stage-3, Stage-6
Stage-2 depends on stages: Stage-0
Stage-3
Hi
Can you please post explain plan for your insert query? I suspect sorted
dynamic partition optimization is bailing out because of
the constant value for ‘dt' column. If you are not seeing a reducer then its
likely not using the sorted dynamic partition optimization.
You are probably hitting t
reading this:
"but when I add 2000 new titles with 300 rows each"
I'm thinking that you are over-partitioning your data
I'm not sure exactly how that relates to the OOM error you are getting (it may
not)I'd test things out partitioning by date-only maybe date +
title_type, but adding
