file or configuration file which are loading at client.
Thanks & Regards
Rohith Sharma K S
From: Shushant Arora [mailto:shushantaror...@gmail.com]
Sent: 21 April 2015 18:26
To: user@hive.apache.org
Subject: mapred.reduce.tasks
In MapReduce job how reduce tasks numbers are decided ?
I hav
In MapReduce job how reduce tasks numbers are decided ?
I haven't override the mapred.reduce.tasks property and its creating ~700
reduce tasks.
Thanks
Hi, all
Why set mapred.reduce.tasks does not work in hive-0.70-rc1 edition?
The source code follows:
private void setNumberOfReducers() throws IOException {
// this is a temporary hack to fix things that are not fixed in the compiler
Integer numReducersFromWork = work.getNumReduceTasks
Try -1, judging from this:
mapred.reduce.tasks
-1
The default number of reduce tasks per job. Typically set
to a prime close to the number of available hosts. Ignored when
mapred.job.tracker is "local". Hadoop set this to 1 by default, whereas hive
uses -1 as its def
I set mapred.reduce.tasks manually to have a single wave of reducers (does
that make sense, by the way?)
When I save the data, I often end up with a bunch of small files because we
use compression and Hive doesn't seem to merge small compressed files.
So my question is: can I di
