[
https://issues.apache.org/jira/browse/HIVE-21731?focusedWorklogId=243533&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-243533
]
ASF GitHub Bot logged work on HIVE-21731:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 16/May/19 18:20
Start Date: 16/May/19 18:20
Worklog Time Spent: 10m
Work Description: maheshk114 commented on pull request #628: HIVE-21731 :
Hive import fails, post upgrade of source 3.0 cluster, to a target 4.0 cluster
with strict managed table set to true.
URL: https://github.com/apache/hive/pull/628#discussion_r284828193
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/load/table/LoadTable.java
##########
@@ -159,10 +159,20 @@ public TaskTracker tasks() throws Exception {
return tracker;
}
- private ReplLoadOpType getLoadTableType(Table table) throws
InvalidOperationException, HiveException {
+ private ReplLoadOpType getLoadTableType(Table table, boolean
isBootstrapDuringInc)
+ throws InvalidOperationException, HiveException {
if (table == null) {
return ReplLoadOpType.LOAD_NEW;
}
+
+ // In case user has asked for bootstrap of transactional table, we replace
the old one if present. This is to
+ // make sure that the transactional info like write id etc for the table
is consistent between the
+ // source and target cluster.
+ if (isBootstrapDuringInc && AcidUtils.isTransactionalTable(table)) {
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 243533)
Time Spent: 1h 40m (was: 1.5h)
> Hive import fails, post upgrade of source 3.0 cluster, to a target 4.0
> cluster with strict managed table set to true.
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-21731
> URL: https://issues.apache.org/jira/browse/HIVE-21731
> Project: Hive
> Issue Type: Bug
> Reporter: mahesh kumar behera
> Assignee: mahesh kumar behera
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21731.01.patch, HIVE-21731.02.patch
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> The scenario is
> # Replication policy is set with hive 3.0 source cluster (strict managed
> table set to false) and hive 4.0 target cluster with strict managed table set
> true.
> # User upgrades the 3.0 source cluster to 4.0 cluster using upgrade tool.
> # The upgrade converts all managed tables to acid tables.
> # In the next repl dump, user sets hive .repl .dump .include .acid .tables
> and hive .repl .bootstrap. acid. tables set true triggering bootstrap of
> newly converted ACID tables.
> # As the old tables are non-txn tables, dump is not filtering the events
> even tough bootstrap acid table is set to true. This is causing the repl load
> to fail as the write id is not set in the table object.
> # If we ignore the event replay, the bootstrap is failing with dump
> directory mismatch error.
> The fix should be
> # Ignore dumping the alter table event if bootstrap acid table is set true
> and the alter is converting a non-acid table to acid table.
> # In case of bootstrap during incremental load, ignore the dump directory
> property set in table object.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
