Re: Hive Mapred local task distribution

2014-09-06 Thread Xuefu Zhang
You might be able to control what tasks to run locally. However, once they run locally, they have to do so on HiveServer2 host. It's possible to run the local tasks in separate JVMs. Still, the same host. --Xuefu On Sat, Sep 6, 2014 at 7:51 AM, Abhilash L L wrote: > Hi Xuefu, > > Yea, cur

Re: Hive Mapred local task distribution

2014-09-06 Thread Abhilash L L
Hi Xuefu, Yea, currently we have only one HiveServer2 host where the map red local tasks run. Any other solution other than LBing it ? Regards, Abhilash L L Capillary Technologies M:919886208262 abhil...@capillarytech.com | www.capillarytech.com Email from people at capillarytech.com ma

Re: Hive Mapred local task distribution

2014-09-06 Thread Xuefu Zhang
By "same host", don't you mean your HiveServer2 host? One solution is to have multiple HiveServer2 instances and do load balance among them. --Xuefu On Fri, Sep 5, 2014 at 11:37 PM, Abhilash L L wrote: > Hello, > >We are using Hive 0.11 connecting to it via Hive Thrift server 2. > >A l

Hive Mapred local task distribution

2014-09-05 Thread Abhilash L L
Hello, We are using Hive 0.11 connecting to it via Hive Thrift server 2. A lot of our queries are launching map red local tasks, which is good and expected. Since we are firing queries in parallel, these tasks are all starting on same host and consuming a lot of resources. Is there a w