[
https://issues.apache.org/jira/browse/HIVE-14323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15392146#comment-15392146
]
Rajesh Balamohan commented on HIVE-14323:
-----------------------------------------
[~cdrome] - Sure, you can take over the changes in
FileOutputCommitterContainer. Rest of the changes are specific to #1 mentioned
in the JIRA and certain lookups (like delete() followed by exists()) which turn
out to be expensive in S3A. These wouldn't show up in HDFS as these ops are
fairly inexpensive compared to S3A.
> Reduce number of FS permissions and redundant FS operations
> -----------------------------------------------------------
>
> Key: HIVE-14323
> URL: https://issues.apache.org/jira/browse/HIVE-14323
> Project: Hive
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Priority: Minor
> Attachments: HIVE-14323.1.patch
>
>
> Some examples are given below.
> 1. When creating stage directory, FileUtils sets the directory permissions by
> running a set of chgrp and chmod commands. In systems like S3, this would not
> be relevant.
> 2. In some cases, fs.delete() is followed by fs.exists(). In this case, it
> might be redundant to check for exists() (lookup ops are expensive in systems
> like S3).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
