[
https://issues.apache.org/jira/browse/HIVE-26319?focusedWorklogId=785157&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-785157
]
ASF GitHub Bot logged work on HIVE-26319:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 27/Jun/22 15:55
Start Date: 27/Jun/22 15:55
Worklog Time Spent: 10m
Work Description: kasakrisz commented on code in PR #3362:
URL: https://github.com/apache/hive/pull/3362#discussion_r907541673
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##########
@@ -858,19 +861,25 @@ private static boolean
hasParquetListColumnSupport(Properties tableProps, Schema
* @param overwrite If we have to overwrite the existing table or just add
the new data
* @return The generated JobContext
*/
- private Optional<JobContext> generateJobContext(Configuration configuration,
String tableName, boolean overwrite) {
+ private Optional<List<JobContext>> generateJobContext(Configuration
configuration, String tableName,
+ boolean overwrite) {
JobConf jobConf = new JobConf(configuration);
- Optional<SessionStateUtil.CommitInfo> commitInfo =
SessionStateUtil.getCommitInfo(jobConf, tableName);
- if (commitInfo.isPresent()) {
- JobID jobID = JobID.forName(commitInfo.get().getJobIdStr());
- commitInfo.get().getProps().forEach(jobConf::set);
- jobConf.setBoolean(InputFormatConfig.IS_OVERWRITE, overwrite);
-
- // we should only commit this current table because
- // for multi-table inserts, this hook method will be called sequentially
for each target table
- jobConf.set(InputFormatConfig.OUTPUT_TABLES, tableName);
-
- return Optional.of(new JobContextImpl(jobConf, jobID, null));
+ Optional<Map<String, SessionStateUtil.CommitInfo>> commitInfoMap =
+ SessionStateUtil.getCommitInfo(jobConf, tableName);
+ if (commitInfoMap.isPresent()) {
+ List<JobContext> jobContextList = Lists.newLinkedList();
+ for (SessionStateUtil.CommitInfo commitInfo :
commitInfoMap.get().values()) {
+ JobID jobID = JobID.forName(commitInfo.getJobIdStr());
+ commitInfo.getProps().forEach(jobConf::set);
+ jobConf.setBoolean(InputFormatConfig.IS_OVERWRITE, overwrite);
+
+ // we should only commit this current table because
+ // for multi-table inserts, this hook method will be called
sequentially for each target table
+ jobConf.set(InputFormatConfig.OUTPUT_TABLES, tableName);
+
+ jobContextList.add(new JobContextImpl(jobConf, jobID, null));
+ }
+ return Optional.of(jobContextList);
Review Comment:
changed return type from `Optional<List>` to `List`
Issue Time Tracking
-------------------
Worklog Id: (was: 785157)
Time Spent: 5h 10m (was: 5h)
> Iceberg integration: Perform update split early
> -----------------------------------------------
>
> Key: HIVE-26319
> URL: https://issues.apache.org/jira/browse/HIVE-26319
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
> Reporter: Krisztian Kasa
> Assignee: Krisztian Kasa
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> Extend update split early to iceberg tables like in HIVE-21160 for native
> acid tables
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
