[
https://issues.apache.org/jira/browse/HIVE-11276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14631311#comment-14631311
]
Xuefu Zhang commented on HIVE-11276:
------------------------------------
Thanks. I will create a JIRA for that.
> Optimization around job submission and adding jars [Spark Branch]
> -----------------------------------------------------------------
>
> Key: HIVE-11276
> URL: https://issues.apache.org/jira/browse/HIVE-11276
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Affects Versions: 1.1.0
> Reporter: Xuefu Zhang
> Assignee: Chengxiang Li
>
> It seems that Hive on Spark has some room for performance improvement on job
> submission. Specifically, we are calling refreshLocalResources() for every
> job submission despite there is are no changes in the jar list. Since Hive on
> Spark is reusing the containers in the whole user session, we might be able
> to optimize that.
> We do need to take into consideration the case of dynamic allocation, in
> which new executors might be added.
> This task is some R&D in this area.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
