vineetgarg02 commented on a change in pull request #552: Hive 21279
URL: https://github.com/apache/hive/pull/552#discussion_r260934183
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
##########
@@ -1538,18 +1551,14 @@ public static void mvFileToFinalPath(Path specPath,
Configuration hconf,
// move to the file destination
Utilities.FILE_OP_LOGGER.trace("Moving tmp dir: {} to: {}", tmpPath,
specPath);
-
- perfLogger.PerfLogBegin("FileSinkOperator", "RenameOrMoveFiles");
- if (isBlobStorage) {
- // HIVE-17113 - avoid copying files that may have been written to
the temp dir by runaway tasks,
- // by moving just the files we've tracked from
removeTempOrDuplicateFiles().
- Utilities.moveSpecifiedFiles(fs, tmpPath, specPath, filesKept);
+ if(shouldAvoidRename(conf, hconf)){
+ LOG.debug("Skipping rename/move files. Files to be kept are: " +
filesKept.toString());
+ conf.getFilesToFetch().addAll(filesKept);
Review comment:
Good idea. Let me see if this could be done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
