[
https://issues.apache.org/jira/browse/HIVE-11267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14628011#comment-14628011
]
Xuefu Zhang commented on HIVE-11267:
------------------------------------
[~chengxiang li], thanks for looking into this. I'm not sure if fetching twice
works for all queries as some queries don't have fetch tasks. I'd think your
original idea of writing multiple times (one for each equal leaf work) in the
FileSinkOperator of the leaf work might be better. You can keep a counter in
the operator when you combine equal leaf works. for this purpose. Thanks.
> Combine equavilent leaf works in SparkWork[Spark Branch]
> --------------------------------------------------------
>
> Key: HIVE-11267
> URL: https://issues.apache.org/jira/browse/HIVE-11267
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Reporter: Chengxiang Li
> Assignee: Chengxiang Li
> Priority: Minor
>
> There could be multi leaf works in SparkWork, like self-union query. If the
> subqueries are same with each other, we may combine the subqueries, and just
> execute once, then fetch twice in FetchTask.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
