[ 
https://issues.apache.org/jira/browse/HIVE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16172840#comment-16172840
 ] 

anishek commented on HIVE-17426:
--------------------------------

All tasks which share some sort of threadlocal or other variables which are not 
yet converted to thread safe access are not going to be run in parallel. as you 
pointed out the above tasks do not fall in that category however.

explain : will only be done individually and not generated along with other 
tasks, hence there is no parallelism that can be there. 
fetch : it seems to be using a bunch of Static methods and i didnt go through 
the whole tree to make sure its thread safe. it will also be the last task in 
any execution so didnt try to parallelize it.
repldump : this is again like "explain" 

for replication we will need parallelism for most of the tasks hence i went 
with default as allowed and only few in blacklist.

> Execution framework in hive to run tasks in parallel
> ----------------------------------------------------
>
>                 Key: HIVE-17426
>                 URL: https://issues.apache.org/jira/browse/HIVE-17426
>             Project: Hive
>          Issue Type: Improvement
>          Components: HiveServer2
>    Affects Versions: 3.0.0
>            Reporter: anishek
>            Assignee: anishek
>             Fix For: 3.0.0
>
>         Attachments: HIVE-17426.0.patch, HIVE-17426.1.patch, 
> HIVE-17426.2.patch, HIVE-17426.3.patch, HIVE-17426.4.patch, HIVE-17426.5.patch
>
>
> the execution framework currently only runs MR / Spark  Tasks in parallel 
> when {{set hive.exec.parallel=true}}.
> Allow other types of tasks to run in parallel as well to support replication 
> scenarios in hive. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to