Re: Disable Hive autogather optimization

thanks Mich. I will test this out and get back to you! On Fri, Apr 29, 2016 at 4:42 PM, Mich Talebzadeh wrote: > apologies should read "Udit" > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >

Re: Disable Hive autogather optimization

apologies should read "Udit" Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * http://talebzadehmich.wordpress.com On 30 April 2016 at 00:35, M

Re: Disable Hive autogather optimization

Hi Unit, *For new tables* Disable stats autogathering in Hive when creating a new table and populating it SET hive.stats.autogather=false; *Already existing tables* As a work-around you can try this on the already existing tables by manually alter the numRows to -1 ALTER TABLE PARTITION S

Re: Disable Hive autogather optimization

Hi, Thanks for the replies. We have a scenario where we have an ETL job inserting into a table with thousands of partitions using dynamic partitioning. We have certain SLA's within which we would like the job to finish and sometimes there are scenarios where they are missed (extra data or a busy c

Re: Disable Hive autogather optimization

Hi Udit, Could u be more specific about your problem? Like, what settings you have, what query you run and what is the result and what result do you expect? From what you said, my understanding is that, you want to wipe out the basic stats for existing tables? And, could u also let us kno

Re: Disable Hive autogather optimization

Hi Is this what is detailed in the following Jira Description Hive will collect table stats when set hive.stats.autogather=true during the INSERT OVERWRITE command. And then the users need to collect the column stats themselves using "Analyze" co

Re: Disable Hive autogather optimization

Any insights on this? On Tue, Apr 26, 2016 at 7:32 PM, Udit Mehta wrote: > Update: Realized this works if we create a fresh table with this config > already disabled but does not work if there is already a table created when > this config was enabled. We now need to figure out how to disable thi

Re: Disable Hive autogather optimization

Update: Realized this works if we create a fresh table with this config already disabled but does not work if there is already a table created when this config was enabled. We now need to figure out how to disable this config for a table created when this config was true. On Tue, Apr 26, 2016 at 6

Re: Disable Hive autogather optimization

Hive version we are using is 1.2.1. On Tue, Apr 26, 2016 at 6:01 PM, Udit Mehta wrote: > Hi, > > We need to disable the Hive autogather stats optimization by disabling " > *hive.stats.autogather*" but for some reason, the config change doesnt > seem to go through. We modified this config in the

Disable Hive autogather optimization

Hi, We need to disable the Hive autogather stats optimization by disabling " *hive.stats.autogather*" but for some reason, the config change doesnt seem to go through. We modified this config in the hive-site.xml and restarted the Hive metastore. We also made this change explicitly in the job but