Hey David,
Sure thing. Play around with that property's value, see if that makes any
difference.
Also, if you could search to see if a file with a name like *hive_skew_join_
**bigkeys* exists on HDFS. Perhaps, it's looking at a different path. If
so, we can figure out how to fix that.
Mark
On Mo
On 30 Nov 2012, at 16:46, Mark Grover wrote:
Hi David, It seems like Hive is unable to find the skewed keys on
HDFS. Did you set *hive.skewjoin.key property? If so, to what value?*
Hey Mark,
thanks for answering!
I didn't set it to anything, but left it at its default value (100,000
IIRC). I
Hi David,
It seems like Hive is unable to find the skewed keys on HDFS.
Did you set *hive.skewjoin.key property? If so, to what value?*
Mark
On Fri, Nov 30, 2012 at 2:10 AM, David Morel wrote:
> Hi,
>
> I am trying to solve the "last reducer hangs because of GC because of
> truckloads of data" i
Hi,
I am trying to solve the "last reducer hangs because of GC because of
truckloads of data" issue that I have on some queries, by using SET
hive.optimize.skewjoin=true; Unfortunately, every time I try this, I
encounter an error of the form:
...
2012-11-30 10:42:39,181 Stage-10 map = 100%,
