Re: Disable Hive autogather optimization

Update: Realized this works if we create a fresh table with this config already disabled but does not work if there is already a table created when this config was enabled. We now need to figure out how to disable this config for a table created when this config was true. On Tue, Apr 26, 2016 at 6

Re: Disable Hive autogather optimization

Hive version we are using is 1.2.1. On Tue, Apr 26, 2016 at 6:01 PM, Udit Mehta wrote: > Hi, > > We need to disable the Hive autogather stats optimization by disabling " > *hive.stats.autogather*" but for some reason, the config change doesnt > seem to go through. We modified this config in the

Disable Hive autogather optimization

Hi, We need to disable the Hive autogather stats optimization by disabling " *hive.stats.autogather*" but for some reason, the config change doesnt seem to go through. We modified this config in the hive-site.xml and restarted the Hive metastore. We also made this change explicitly in the job but

Re: Is there a way to resolve Fair Scheduler Problem

Thanks Khaja. I will look into it. On 4/25/2016 4:38 PM, Khaja Hussain wrote: Hi If you need other application in a queue to start you can use pre-emption. Below link has the details. What pre-emption does is it guarantee's capacity, in which case your job will start. If there is no load o

RE: Hive query to split one row into many rows such that Row 1 will have col 1 Name, col 1 Value and Row 2 will have col 2 Name and col 2 value

You are welcome ☺ I’ve tried to guess the requested result for your last question. It can be very helpful if you can create a small example containing your original data and the requested result. Dudu Given the following table, ‘t’: i c1 c2 c3 1 1 12 15 2 1 13 11 3 3 11 13 4 1

RE: Hive query to split one row into many rows such that Row 1 will have col 1 Name, col 1 Value and Row 2 will have col 2 Name and col 2 value

if you are doing group by, you could have potential duplicates on your concat_wstake a look at using collect_set or collect_list. if you do select col_a, collect_set(concat_ws(', ',col_b,col_c)) from t you will have an array of unique collection pairs...collect_list will give you all pairs.

Re: Hive query to split one row into many rows such that Row 1 will have col 1 Name, col 1 Value and Row 2 will have col 2 Name and col 2 value

Thanks a lot Dudu. Could you also tell how can I use concat with group by clause in have. I have n rows with col1, col2, col3 and i want a result grouped by col1 and concat all values of col2 and col3. Id,key,value, value2 __ 1,fname,Dudu, m1 1,lname,Markowitz, m2 2,fname

Hive UDF

Is it possible to run a Hive UDF in Spark DataFrame?