[
https://issues.apache.org/jira/browse/HIVE-24339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263354#comment-17263354
]
Ayush Saxena commented on HIVE-24339:
-------------------------------------
Hi [~abstractdog]
{{hive.exec.parallel.thread.number}} doesn't take affect as part of the
{{With}} clause from repl load. but hive.exec.parallel does; because whether
the task can be run in parallel, is figured out from task conf only :
{code:java}
private TaskRunner launchTask(Task<?> task, boolean noName, String jobName,
int jobCount) throws HiveException {
.
.
.
if (HiveConf.getBoolVar(task.getConf(), HiveConf.ConfVars.EXECPARALLEL) &&
task.canExecuteInParallel()) {
{code}
But the number of tasks to launch is from the driver :
{code:java}
private void launchTasks(boolean noName, int jobCount, String jobName) throws
HiveException {
// Launch upto maxthreads tasks
Task<?> task;
int maxthreads = HiveConf.getIntVar(driverContext.getConf(),
HiveConf.ConfVars.EXECPARALLETHREADNUMBER);
{code}
So, I think this is a correct behaviour, you need to set that value at session
level only.
> REPL LOAD command ignores config properties set by WITH clause
> --------------------------------------------------------------
>
> Key: HIVE-24339
> URL: https://issues.apache.org/jira/browse/HIVE-24339
> Project: Hive
> Issue Type: Bug
> Reporter: László Bodor
> Priority: Major
>
> By debug messages we confirmed that REPL LOAD command ignored some config
> properties when they were provided in WITH clause, e.g.:
> {code}
> REPL LOAD bdpp01pub FROM
> 'hdfs://prdpdp01//apps/hive/repl/8237c7bd-ba26-4425-8659-3a0d32ab312c' WITH
> ('mapreduce.job.queuename'='default','hive.exec.parallel'='true','hive.exec.parallel.thread.number'='128',
> ...
> {code}
> We found that it was working on 16 threads, ignoring
> 'hive.exec.parallel.thread.number'='128'. Setting this property on session
> level worked.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
