Hi
In MapReduce , number of reducers launched is set by property
“mapreduce.job.reduces”. And via java API Job#setNumReduceTasks(int tasks).
In your MR job, somewhere in the program they are setting number of reducer
task using java API or property. May be you can check the MR job or property
> A bit digging shows that GenericUDF#configure method was not called. So
>in this case, is it possible to get counters through other means, or we
>have to implement Counter concept ourselves?
You should be getting a TezContext object there (which inherits from
MapRedContext).
And the method s
We have a UDF to collect some counts during Hive execution. It has been working
fine until tez is enabled.
A bit digging shows that GenericUDF#configure method was not called. So in this
case, is it possible to get counters through other means, or we have to
implement Counter concept ourselves?
What we do is to upload the script first into HDFS, then use the "file" option
in WebHCat to submit the queries - so I think the REST call should not
matter... right?
Xiaoyong
From: Eugene Koifman [mailto:ekoif...@hortonworks.com]
Sent: Monday, April 20, 2015 10:32 PM
To: user@hive.apache.org
S
In MapReduce job how reduce tasks numbers are decided ?
I haven't override the mapred.reduce.tasks property and its creating ~700
reduce tasks.
Thanks
Hi Gopal,
The table created is not a bucketed table, but a dynamic partitioned
table. I took the script test from
https://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/orc_merge7.q
- create table orc_merge5 (userid bigint, string1 string, subtype
double, decimal1 de
>alter table concatenate do not work? I have a dynamic
>partitioned table (stored as orc). I tried to alter concatenate, but it
>did not work. See my test result.
ORC fast concatenate does work on partitioned tables, but it doesn¹t work
on bucketed tables.
Bucketed tables cannot merge files, s
Hi Gopal,
Thanks for your explanation.
What could be the case that SET hive.merge.orcfile.stripe.level=true &&
alter table concatenate do not work? I have a dynamic
partitioned table (stored as orc). I tried to alter concatenate, but it
did not work. See my test result.
hive> SET hive.merg
