[
https://issues.apache.org/jira/browse/HIVE-24675?focusedWorklogId=540896&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-540896
]
ASF GitHub Bot logged work on HIVE-24675:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 25/Jan/21 06:58
Start Date: 25/Jan/21 06:58
Worklog Time Spent: 10m
Work Description: ArkoSharma commented on a change in pull request #1898:
URL: https://github.com/apache/hive/pull/1898#discussion_r563496522
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplExternalTables.java
##########
@@ -183,6 +184,22 @@ private void dirLocationToCopy(FileList fileList, Path
sourcePath, HiveConf conf
throws HiveException {
Path basePath = getExternalTableBaseDir(conf);
Path targetPath = externalTableDataPath(conf, basePath, sourcePath);
+ //Here, when src and target are HA clusters with same NS, then
sourcePath would have the correct host
+ //whereas the targetPath would have an host that refers to the target
cluster. This is fine for
+ //data-copy running during dump as the correct logical locations would
be used. But if data-copy runs during
+ //load, then the remote location needs to point to the src cluster from
where the data would be copied and
+ //the common original NS would suffice for targetPath.
+
if(hiveConf.getBoolVar(HiveConf.ConfVars.REPL_HA_DATAPATH_REPLACE_REMOTE_NAMESERVICE)
&&
+
hiveConf.getBoolVar(HiveConf.ConfVars.REPL_RUN_DATA_COPY_TASKS_ON_TARGET)) {
+ String remoteNS =
hiveConf.get(HiveConf.ConfVars.REPL_HA_DATAPATH_REPLACE_REMOTE_NAMESERVICE_NAME.varname);
+ if (StringUtils.isEmpty(remoteNS)) {
+ throw new SemanticException(ErrorMsg.REPL_INVALID_CONFIG_FOR_SERVICE
Review comment:
Yes, it is a non recoverable error if the remote NS config is not
specified.There needs to be a valid remote NS name and the same name should be
used in both the dump and load commands.
The same exception is being thrown for managed tables.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 540896)
Time Spent: 50m (was: 40m)
> Handle external table replication for HA with same NS and lazy copy.
> --------------------------------------------------------------------
>
> Key: HIVE-24675
> URL: https://issues.apache.org/jira/browse/HIVE-24675
> Project: Hive
> Issue Type: Bug
> Reporter: Arko Sharma
> Assignee: Arko Sharma
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-24675.01.patch, HIVE-24675.02.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
