[
https://issues.apache.org/jira/browse/HIVE-8622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14202496#comment-14202496
]
Suhas Satish commented on HIVE-8622:
------------------------------------
[~csun] - We already have a mapr of BaseWork containing the map-join to its
parent ReduceSinks.
This exists as {{linkWorkWithReduceSinkMap}} in {{GenSparkProcContext}}
Do you think we can leverage that in some way, or replace the RSs in that Map
with the HashTableSinks that we introduced? It looks like we should still
propagate the whole GenSparkProcContext to the {{SparkMapJoinResolver}} through
the SparkCompiler.generateTaskTree(...) and {{SparkCompiler.optimizeTaskPlan}}
All the state information stored there will make life a lot easier.
> Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]
> ----------------------------------------------------------------
>
> Key: HIVE-8622
> URL: https://issues.apache.org/jira/browse/HIVE-8622
> Project: Hive
> Issue Type: Sub-task
> Reporter: Suhas Satish
> Assignee: Chao
> Attachments: HIVE-8622.2-spark.patch, HIVE-8622.3-spark.patch,
> HIVE-8622.patch
>
>
> This is a sub-task of map-join for spark
> https://issues.apache.org/jira/browse/HIVE-7613
> This can use the baseline patch for map-join
> https://issues.apache.org/jira/browse/HIVE-8616
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
