[
https://issues.apache.org/jira/browse/HIVE-21471?focusedWorklogId=217554&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-217554
]
ASF GitHub Bot logged work on HIVE-21471:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 23/Mar/19 03:59
Start Date: 23/Mar/19 03:59
Worklog Time Spent: 10m
Work Description: sankarh commented on pull request #578: HIVE-21471:
Replicating conversion of managed to external table leaks HDFS files at target.
URL: https://github.com/apache/hive/pull/578#discussion_r268381165
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/CreateTableOperation.java
##########
@@ -54,17 +55,24 @@ public int execute() throws HiveException {
Table tbl = desc.toTable(context.getConf());
LOG.debug("creating table {} on {}", tbl.getFullyQualifiedName(),
tbl.getDataLocation());
- if (desc.getReplicationSpec().isInReplicationScope() &&
(!desc.getReplaceMode())){
- // if this is a replication spec, then replace-mode semantics might
apply.
- // if we're already asking for a table replacement, then we can skip
this check.
- // however, otherwise, if in replication scope, and we've not been
explicitly asked
- // to replace, we should check if the object we're looking at exists,
and if so,
+ boolean dataLocationChanged = false;
+ if (desc.getReplicationSpec().isInReplicationScope()) {
+ // If in replication scope, we should check if the object we're looking
at exists, and if so,
// trigger replace-mode semantics.
Table existingTable = context.getDb().getTable(tbl.getDbName(),
tbl.getTableName(), false);
- if (existingTable != null){
+ if (existingTable != null) {
if
(desc.getReplicationSpec().allowEventReplacementInto(existingTable.getParameters()))
{
desc.setReplaceMode(true); // we replace existing table.
ReplicationSpec.copyLastReplId(existingTable.getParameters(),
tbl.getParameters());
+
+ // If location of an existing managed table is changed, then need to
delete the old location if exists.
+ // This scenario occurs when a managed table is converted into
external table at source. In this case,
+ // at target, the table data would be moved to different location
under base directory for external tables.
+ if (existingTable.getTableType().equals(TableType.MANAGED_TABLE)
+ && tbl.getTableType().equals(TableType.EXTERNAL_TABLE)
+ &&
(!existingTable.getDataLocation().equals(tbl.getDataLocation()))) {
Review comment:
It's not possible in replication flow. But I kept the check explicitly for
better readability. There is no harm in keeping it. If you insist, let me
change it to assert.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 217554)
Time Spent: 2.5h (was: 2h 20m)
> Replicating conversion of managed to external table leaks HDFS files at
> target.
> -------------------------------------------------------------------------------
>
> Key: HIVE-21471
> URL: https://issues.apache.org/jira/browse/HIVE-21471
> Project: Hive
> Issue Type: Bug
> Components: repl
> Affects Versions: 4.0.0
> Reporter: Sankar Hariappan
> Assignee: Sankar Hariappan
> Priority: Major
> Labels: DR, pull-request-available, replication
> Attachments: HIVE-21471.01.patch, HIVE-21471.02.patch
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> While replicating the ALTER event to convert managed table to external table,
> the data location for the table is changed under input base directory for
> external tables replication. But, the old location remains there and would be
> leaked for ever.
> ALTER TABLE T1 SET TBLPROPERTIES('EXTERNAL'='true');
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
