The queue specific configurations are not hive client specific, they have to be
configured on JobTracker before JT is started up. All the Hive Cli should try
setting is which queue they will want the DAG from hive query to be submitted
to.
So your capacity-scheduler.xml in $HADOOP_CONF_DIR sho
Hi Sreekanth,
When you mention about setting the max task limit, do you mean by executing
set mapred.capacity-scheduler.queue..maximum-capacity = ?
Is it only available on hadoop 0.21?
Thanks,
Rosanna
On 5/1/11 8:42 PM, "Sreekanth Ramakrishnan" wrote:
>
> The design goal of CapacitySchedul
The design goal of CapacityScheduler is maximizing the utilization of cluster
resources but it does not fairly allocate the share amongst the total number of
users present in the system.
The user limit states the number of concurrent users who can use the slots in
the queue. But then these lim
Hi Sreekanth,
Thank you very much for your clarification. Setting the max task limits on
queues will work but can we do something on the max user limit? Is it
pre-emptible also? We are exploring about the possibility of running the
queries with different users for capacity scheduler to maximize th
Hi
Currently CapacityScheduler does not have pre-emption. So basically when the
Job1 starts finishing and freeing up the Job2's tasks will start getting
scheduled. One way you can prevent that queue capacities are not elastic in
nature is by setting max task limits on queues. That way your job1
