[
https://issues.apache.org/jira/browse/HIVE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gang Tim Liu updated HIVE-3251:
-------------------------------
Status: Patch Available (was: In Progress)
https://reviews.facebook.net/D4089
> Hive doesn't remove scrach directories while killing running MR job
> -------------------------------------------------------------------
>
> Key: HIVE-3251
> URL: https://issues.apache.org/jira/browse/HIVE-3251
> Project: Hive
> Issue Type: Bug
> Components: Server Infrastructure
> Reporter: Gang Tim Liu
> Assignee: Gang Tim Liu
>
> While killing running MR job, hive doesn't clean up scratch directory
> (mapred.cache.files). So that, afterwards, scratch directory is left there in
> hdfs. HDFS name node doesn't know it and try to do lease recovery. while such
> instances happen more, it will eventually crash namenode.
> The fix is to leverage hdfs clean up functionality. While creating scratch
> dirs, hive registers it to hdfs cleanup hook. While killing happens, hdfs
> will clean them up.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
