[
https://issues.apache.org/jira/browse/HIVE-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-4679:
---------------------------------
Status: Patch Available (was: Open)
> WebHCat can deadlock Hadoop if the number of concurrently running tasks if
> higher or equal than the number of mappers
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-4679
> URL: https://issues.apache.org/jira/browse/HIVE-4679
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
> Affects Versions: 0.10.0
> Reporter: Eugene Koifman
> Assignee: Eugene Koifman
> Attachments: HIVE-4679.patch
>
>
> o In the current Templeton design, each time a Job is submitted thru the REST
> API (it can be Pig/Hive or MR job), it will consume one Hadoop map slot.
> Given that the number of map slots is finite in the cluster (16 node cluster
> will have 32 map slots), in some circumstances, a user can deadlock the
> cluster if Templeton job submission pipeline takes over all map slots
> (Templeton map tasks will wait for the actual underlying jobs to complete,
> what will never happen, given that Hadoop has no free map slots to schedule
> new tasks).
> o HCat queries use a different mechanism and do not contribute to the
> deadlock.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
