Hi, I'm trying to set hive.exec.mode.local.auto.inputbytes.max & hive.exec.mode.local.auto.tasks.max to 1 or 0 but still local mode is being used instead of M/R.
Any ideas? Thank you. Daniel On Thu, Sep 3, 2015 at 8:02 AM, sreebalineni . <sreebalin...@gmail.com> wrote: > Hi, > > Is not it that you should set it true, by default it is disabled which is > false. > > Hive analyzes the size of each map-reduce job in a query and may run it > locally if the following thresholds are satisfied: > > - The total input size of the job is lower than: > hive.exec.mode.local.auto.inputbytes.max (128MB by default) > - The total number of map-tasks is less than: > hive.exec.mode.local.auto.tasks.max (4 by default) > - The total number of reduce tasks required is 1 or 0. > > So for queries over small data sets, or for queries with multiple > map-reduce jobs where the input to subsequent jobs is substantially smaller > (because of reduction/filtering in the prior job), jobs may be run locally. > > so we may need to check the sizeof your input, which version of hive are > you using? it can work only from Hive 0.7 onwards > > On Wed, Sep 2, 2015 at 4:46 PM, Daniel Haviv < > daniel.ha...@veracity-group.com> wrote: > >> Hi, >> I would like to disable the optimization where a query that just selects >> data is running without mapreduce (local mode). >> >> hive.exec.mode.local.auto is set to false but hive still runs in local mode >> for some queries. >> >> >> How can I disable local mode completely? >> >> >> Thank you. >> >> Daniel >> >> >
