[
https://issues.apache.org/jira/browse/HIVE-20967?focusedWorklogId=238979&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-238979
]
ASF GitHub Bot logged work on HIVE-20967:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 08/May/19 04:10
Start Date: 08/May/19 04:10
Worklog Time Spent: 10m
Work Description: maheshk114 commented on pull request #613: HIVE-20967
URL: https://github.com/apache/hive/pull/613#discussion_r281906378
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
##########
@@ -805,22 +811,89 @@ public Partition alterPartition(RawStore msdb, Warehouse
wh, String catName, Str
return oldParts;
}
- private void checkTableTypeConversion(Database db, Table oldTbl, Table
newTbl)
+ // Validate changes to partition's location to protect against errors on
migration during
+ // replication
+ private void blockPartitionLocationChangesOnReplSource(Database db, Table
tbl,
+ EnvironmentContext ec)
+ throws InvalidOperationException {
+ // If the database is not replication source, nothing to do
+ if (!ReplChangeManager.isSourceOfReplication(db)) {
+ return;
+ }
+
+ // For now validate only the changes when strict managed tables is false.
That's when there's
+ // scope for migration during replication, at least for now.
+ if
(conf.getBoolean(MetastoreConf.ConfVars.STRICT_MANAGED_TABLES.getHiveName(),
false)) {
+ return;
+ }
+
+ // Do not allow changing location of a managed table as as alter event
doesn't capture the
+ // new files list. So, it may cause data inconsistency.
+ boolean isChangingLocation = false;
+ if (ec.isSetProperties()) {
+ String alterType = ec.getProperties().get(ALTER_TABLE_OPERATION_TYPE);
+ if (alterType != null && alterType.equalsIgnoreCase(ALTERLOCATION)) {
Review comment:
add " &&
tbl.getTableType().equalsIgnoreCase(TableType.MANAGED_TABLE.name() . " and
throw exception
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 238979)
Time Spent: 50m (was: 40m)
> Handle alter events when replicate to cluster with hive.strict.managed.tables
> enabled.
> --------------------------------------------------------------------------------------
>
> Key: HIVE-20967
> URL: https://issues.apache.org/jira/browse/HIVE-20967
> Project: Hive
> Issue Type: Sub-task
> Components: repl
> Affects Versions: 4.0.0
> Reporter: mahesh kumar behera
> Assignee: Ashutosh Bapat
> Priority: Minor
> Labels: DR, pull-request-available
> Attachments: HIVE-20967.01.patch, HIVE-20967.03.patch,
> HIVE-20967.03.patch, HIVE-20967.04.patch, HIVE-21678.02.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Some of the events from Hive2 may cause conflicts in Hive3
> (hive.strict.managed.tables=true) when applied. So, need to handle them
> properly.
> 1. Alter table to convert non-acid to acid.
> - Do not allow this conversion on source of replication if strict.managed is
> false.
> 2. Alter table or partition that changes the location.
> - For managed tables at source, the table location shouldn't be changed for
> the given non-partitioned table and partition location shouldn't be changed
> for partitioned table as alter event doesn't capture the new files list. So,
> it may cause data inconsistsency. So, if database is enabled for replication
> at source, then alter location on managed tables should be blocked.
> - For external partitioned tables, if location is changed at source, the the
> location should be changed for the table and any partitions which reside
> within the table location, but not for the partitions which are not within
> the table location. (may be we just need the test).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
