[
https://issues.apache.org/jira/browse/HIVE-21500?focusedWorklogId=226152&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-226152
]
ASF GitHub Bot logged work on HIVE-21500:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Apr/19 16:58
Start Date: 11/Apr/19 16:58
Worklog Time Spent: 10m
Work Description: ashutosh-bapat commented on pull request #591:
HIVE-21500: Disable conversion of managed table to external and vice versa at
source.
URL: https://github.com/apache/hive/pull/591#discussion_r274508055
##########
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplScenariosWithStrictManaged.java
##########
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.parse;
+
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.metastore.conf.MetastoreConf;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * TestReplScenariosWithStrictManaged - Test all replication scenarios with
strict managed enabled
+ * at source and target.
+ */
+public class TestReplScenariosWithStrictManaged extends
BaseReplicationAcrossInstances {
+
+ @BeforeClass
+ public static void classLevelSetup() throws Exception {
+ Map<String, String> overrides = new HashMap<>();
+ overrides.put(HiveConf.ConfVars.HIVE_STRICT_MANAGED_TABLES.varname,
"true");
+ overrides.put(MetastoreConf.ConfVars.CREATE_TABLES_AS_ACID.getHiveName(),
"true");
+ overrides.put(HiveConf.ConfVars.HIVE_SUPPORT_CONCURRENCY.varname, "true");
+ overrides.put(HiveConf.ConfVars.HIVE_TXN_MANAGER.varname,
"org.apache.hadoop.hive.ql.lockmgr.DbTxnManager");
+
+ internalBeforeClassSetup(overrides,
TestReplScenariosWithStrictManaged.class);
+ }
+
+ @Test
+ public void dynamicallyConvertManagedToExternalTable() throws Throwable {
+ // All tables are automatically converted to ACID tables when strict
managed is enabled.
+ // Also, it is not possible to convert ACID table to external table.
+ primary.run("use " + primaryDbName)
+ .run("create table t1 (id int) stored as orc")
+ .run("insert into table t1 values (1)")
+ .run("create table t2 (id int) partitioned by (key int) stored as
orc")
+ .run("insert into table t2 partition(key=10) values (1)")
+ .runFailure("alter table t1 set tblproperties('EXTERNAL'='true')")
+ .runFailure("alter table t2 set tblproperties('EXTERNAL'='true')");
+ }
+
+ @Test
+ public void dynamicallyConvertExternalToManagedTable() throws Throwable {
+ // With Strict managed enabled, it is not possible to convert external
table to ACID table.
+ primary.run("use " + primaryDbName)
+ .run("create external table t1 (id int) stored as orc")
+ .run("insert into table t1 values (1)")
+ .run("create external table t2 (place string) partitioned by
(country string)")
+ .run("insert into table t2 partition(country='india') values
('bangalore')")
+ .runFailure("alter table t1 set tblproperties('EXTERNAL'='false')")
+ .runFailure("alter table t2 set
tblproperties('EXTERNAL'='false')");
Review comment:
Test this and also explicit conversion to transactional table, if it's
possible?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 226152)
Time Spent: 40m (was: 0.5h)
> Disable conversion of managed table to external and vice versa at source.
> -------------------------------------------------------------------------
>
> Key: HIVE-21500
> URL: https://issues.apache.org/jira/browse/HIVE-21500
> Project: Hive
> Issue Type: Sub-task
> Affects Versions: 4.0.0
> Reporter: Sankar Hariappan
> Assignee: Sankar Hariappan
> Priority: Major
> Labels: DR, Replication, pull-request-available
> Attachments: HIVE-21500.01.patch, HIVE-21500.02.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Couple of scenarios for Hive2 to Hive3(strict managed tables enabled)
> replication where managed table is converted to external at source.
> *Scenario-1: (ACID/MM table converted to external at target)*
> 1. Create non-ACID ORC format table.
> 2. Insert some rows
> 3. Replicate this create event which creates ACID table at target (due to
> migration rule). Each insert event adds transactional metadata in HMS
> corresponding to the current table.
> 4. Convert table to external table using ALTER command at source.
> *Scenario-2: (External table at target changes table location)*
> 1. Create non-ACID avro format table.
> 2. Insert some rows
> 3. Replicate this create event which creates external table at target (due to
> migration rule). The data path is chosen under default external warehouse
> directory.
> 4. Convert table to external table using ALTER command at source.
> It is unable to convert an ACID table to external table at target. Also, it
> is hard to detect what would be the table type at target when perform this
> ALTER table operation at source.
> So, it is decided to disable conversion of managed table at source (Hive2) to
> EXTERNAL or vice-versa if the DB is enabled for replication and strict
> managed is disabled.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
