[
https://issues.apache.org/jira/browse/HIVE-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15733460#comment-15733460
]
Sahil Takiar commented on HIVE-15385:
-------------------------------------
I think HIVE-7015 has some info for why failure to set permissions shouldn't
result in failures. Here is the failure that I am hitting:
HDFS permission inheritance follows the BSD rule: when a new file is created,
it inherits the group of the parent directory. There are situations in Hive
when trying to inherit the group of the parent can fail, failure to set the
group can occur in the following situation:
1: {{user1}} uploads a file into HDFS directory {{/test1/}} where {{/test1/}}
has group {{user1}}
2: The {{/user/hive/warehouse}} directory was created by the {{hive}} user and
has group {{hive}}
3: {{user1}} runs a query that loads data from {{/test1/}} and writes it to
{{/user/hive/warehouse/}}
4: The problem is that when the data is moved from {{/test1/}} to
{{/user/hive/warehouse/}}, it retains its original group {{user1}}
5: Hive then tries to set the group to {{hive}} in order to preserve permission
inheritance, but this fails because the query is running as {{user1}}, but
{{user1}} is not part of the {{hive}} group and is not allowed to change the
group of a file to {{hive}}
I think the {{chgrp}} logic is similar to other standard POSIX systems.
I think there can be other situations too, but I'm not an expert in this area
so I would have to play around with it some more.
I've confirmed that everywhere this method is used inherit perms is true,
except for one place: {{DDLTask.truncateTable}} (which looks like it should be
checking if inherit perms is true).
Most of the invocations of {{setFullFileStatus}} are wrapped by a try-catch
block which just logs the exception. There are only a few place where they are
propagated up to the caller, these were all introduced by HIVE-12988,
HIVE-13716, and HIVE-13933 - seemingly by accident.
> Failure to inherit permissions when running HdfsUtils.setFullFileStatus(...,
> false) causes queries to fail
> ----------------------------------------------------------------------------------------------------------
>
> Key: HIVE-15385
> URL: https://issues.apache.org/jira/browse/HIVE-15385
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Attachments: HIVE-15385.1.patch, HIVE-15385.2.patch
>
>
> According to
> https://cwiki.apache.org/confluence/display/Hive/Permission+Inheritance+in+Hive,
> failure to inherit permissions should not cause queries to fail.
> It looks like this was the case until HIVE-13716, which added some code to
> use {{fs.setOwner}}, {{fs.setAcl}}, and {{fs.setPermission}} to set
> permissions instead of shelling out and running {{-chgrp -R ...}}.
> When shelling out, the return status of each command is ignored, so if there
> are any failures when inheriting permissions, a warning is logged, but the
> query still succeeds.
> However, when invoked the {{FileSystem}} API, any failures will be propagated
> up to the caller, and the query will fail.
> This is problematic because {{setFulFileStatus}} shells out when the
> {{recursive}} parameter is set to {{true}}, and when it is false it invokes
> the {{FileSystem}} API. So the behavior is inconsistent depending on the
> value of {{recursive}}.
> We should decide whether or not permission inheritance should fail queries or
> not, and then ensure the code consistently follows that decision.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
