[
https://issues.apache.org/jira/browse/HIVE-25303?focusedWorklogId=648949&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648949
]
ASF GitHub Bot logged work on HIVE-25303:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Sep/21 00:21
Start Date: 10/Sep/21 00:21
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on a change in pull
request #2442:
URL: https://github.com/apache/hive/pull/2442#discussion_r705807353
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java
##########
@@ -472,6 +474,28 @@ private void setLoadFileLocation(
loc = cmv.getLocation();
}
Path location = (loc == null) ? getDefaultCtasLocation(pCtx) : new
Path(loc);
+ if (pCtx.getQueryProperties().isCTAS()) {
+ boolean isExternal = pCtx.getCreateTable().isExternal();
+ boolean isAcid = pCtx.getCreateTable().getTblProps().getOrDefault(
+ hive_metastoreConstants.TABLE_IS_TRANSACTIONAL,
"false").equalsIgnoreCase("true") ||
+
pCtx.getCreateTable().getTblProps().containsKey(hive_metastoreConstants.TABLE_TRANSACTIONAL_PROPERTIES);
+ if ((HiveConf.getBoolVar(conf,
HiveConf.ConfVars.CREATE_TABLE_AS_EXTERNAL) || isExternal) && !isAcid) {
Review comment:
Two reasons I don't want to remove this condition.
1) If we can have this condition we can avoid calling the HMS dry run api,
if the tables are managed tables (which is the current behavior).
2) Only HS2 would know if hive.create.as.external.legacy config, so this
condition should be in HS2
3) L#486 is introducing buckerting_version table parameter in the explain
plan for all the tables. Even if I hide it in L#492 this paramter would be
missing in the describe/explain table commands. It is failing few tests.
So I think this should be ok leaving this condition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 648949)
Time Spent: 2h 50m (was: 2h 40m)
> CTAS hive.create.as.external.legacy tries to place data files in managed WH
> path
> --------------------------------------------------------------------------------
>
> Key: HIVE-25303
> URL: https://issues.apache.org/jira/browse/HIVE-25303
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Standalone Metastore
> Reporter: Sai Hemanth Gantasala
> Assignee: Sai Hemanth Gantasala
> Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> Under legacy table creation mode (hive.create.as.external.legacy=true), when
> a database has been created in a specific LOCATION, in a session where that
> database is Used, tables are created using the following command:
> {code:java}
> CREATE TABLE <tablename> AS SELECT <select statement>{code}
> should inherit the HDFS path from the database's location. Instead, Hive is
> trying to write the table data into
> /warehouse/tablespace/managed/hive/<database_directory_name>/<table_name>
> +Design+:
> In the CTAS query, first data is written in the target directory (which
> happens in HS2) and then the table is created(This happens in HMS). So here
> two decisions are being made i) target directory location ii) how the table
> should be created (table type, sd e.t.c).
> When HS2 needs a target location that needs to be set, it'll make create
> table dry run call to HMS (where table translation happens) and i) and ii)
> decisions are made within HMS and returns table object. Then HS2 will use
> this location set by HMS for placing the data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
