[
https://issues.apache.org/jira/browse/HIVE-14323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15391843#comment-15391843
]
Chris Drome commented on HIVE-14323:
------------------------------------
[~rajesh.balamohan], I just finished working on a patch for
FileOutputCommitterContainer.java which deals with this more thoroughly, as
there are a bunch of fundamental problems at the core of this which lead to
these and other issues, all of which I address.
I might be mistaken, but it appears that your patch also deals with the Hive
query side as well.
Would you mind if I took over the changes to FileOutputCommitterContainer.java
as part of my patch?
Otherwise, could you provide more details about what your patch fixes exactly?
> Reduce number of FS permissions and redundant FS operations
> -----------------------------------------------------------
>
> Key: HIVE-14323
> URL: https://issues.apache.org/jira/browse/HIVE-14323
> Project: Hive
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Priority: Minor
> Attachments: HIVE-14323.1.patch
>
>
> Some examples are given below.
> 1. When creating stage directory, FileUtils sets the directory permissions by
> running a set of chgrp and chmod commands. In systems like S3, this would not
> be relevant.
> 2. In some cases, fs.delete() is followed by fs.exists(). In this case, it
> might be redundant to check for exists() (lookup ops are expensive in systems
> like S3).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
