[
https://issues.apache.org/jira/browse/HIVE-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13544561#comment-13544561
]
Phabricator commented on HIVE-3431:
-----------------------------------
ashutoshc has requested changes to the revision "HIVE-3431 [jira] Resources on
non-local file system should be downloaded to temporary directory sometimes".
Mostly looks good except for unwarranted change of retrieving user name from
env instead of current System properties.
I also realized while reviewing this that we never delete these resources.
This may become a problem for HiveServer since in server mode, various user
files might get downloaded in this server dir which will never be deleted and
thus filling up server disk. I think solution here will be to keep track of
these files in SessionState and delete all such files once the query ends. This
is pre-existing problem which we need to deal with. Can you file a new jira for
this, so that we can follow this up?
INLINE COMMENTS
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:190 Last line of
http://docs.oracle.com/javase/tutorial/essential/environment/env.html
says :
To maximize portability, never refer to an environment variable when the same
value is available in a system property. For example, if the operating system
provides a user name, it will always be available in the system property
user.name.
So, I think change is unwarranted. Please, revert it.
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:943 Per previous
comment, this new method than will no longer be needed.
conf/hive-default.xml.template:1624 Update this also, once you revert
previous change.
REVISION DETAIL
https://reviews.facebook.net/D5199
BRANCH
DPAL-1872
To: JIRA, ashutoshc, navis
Cc: mgrover
> Resources on non-local file system should be downloaded to temporary
> directory sometimes
> ----------------------------------------------------------------------------------------
>
> Key: HIVE-3431
> URL: https://issues.apache.org/jira/browse/HIVE-3431
> Project: Hive
> Issue Type: Improvement
> Components: Configuration
> Affects Versions: 0.10.0
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3431.1.patch.txt, HIVE-3431.D5199.2.patch,
> HIVE-3431.D5199.3.patch
>
>
> "add resource <remote-uri>" command downloads the resource file to location
> specified by conf "hive.downloaded.resources.dir" in local file system. But
> when the command above is executed concurrently to hive-server for same file,
> some client fails by VM crash, which is caused by overwritten file by other
> requests.
> So there should be a configuration to provide per request location for add
> resource command, something like "set
> hiveconf:hive.downloaded.resources.dir=temporary"
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
