[
https://issues.apache.org/jira/browse/HIVE-13572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15257765#comment-15257765
]
Rui Li commented on HIVE-13572:
-------------------------------
Thanks [~ashutoshc] for the suggestion. The v2 patch sets status using
threadpool, which provided better performance in my local tests.
I'll come up with another patch to do it in FS. One concern is that we may
unnecessarily set the permission on intermediate data this way. We can decide
which way to go when the patch is ready.
> Redundant setting full file status in Hive::copyFiles
> -----------------------------------------------------
>
> Key: HIVE-13572
> URL: https://issues.apache.org/jira/browse/HIVE-13572
> Project: Hive
> Issue Type: Bug
> Reporter: Rui Li
> Assignee: Rui Li
> Attachments: HIVE-13572.1.patch, HIVE-13572.2.patch
>
>
> We set full file status in each copy-file thread. I think it's redundant and
> hurts performance when we have multiple files to copy.
> {code}
> if (inheritPerms) {
> ShimLoader.getHadoopShims().setFullFileStatus(conf,
> fullDestStatus, destFs, destf);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
