[
https://issues.apache.org/jira/browse/HIVE-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167022#comment-16167022
]
Zoltan Haindrich edited comment on HIVE-15467 at 9/14/17 10:00 PM:
-------------------------------------------------------------------
okay...after setting yarn to debug and going thru the logs; I found the reason:
nodemanagers are in-operable in case disk usage is over 90%
https://stackoverflow.com/questions/29131449/why-does-hadoop-report-unhealthy-node-local-dirs-and-log-dirs-are-bad
it seems like the llap mini driver doesn't react to "my" extra yarn
settings...so I've moved away some files..from my work disk - and it now works
...someone might disable this feature by setting
{{yarn.nodemanager.disk-health-checker.enable}} to {{false}}...
was (Author: kgyrtkirk):
okay...after setting yarn to debug and crawling to the logs; I found the
reason: nodemanagers are in-operable in case disk usage is over 90%
https://stackoverflow.com/questions/29131449/why-does-hadoop-report-unhealthy-node-local-dirs-and-log-dirs-are-bad
it seems like the llap mini driver doesn't react to "my" extra yarn
settings...so I've moved away some files..from my work disk - and it now works
...someone might disable this feature by setting
{{yarn.nodemanager.disk-health-checker.enable}} to {{false}}...
> escape1.q hangs in TestMiniLlapLocalCliDriver
> ---------------------------------------------
>
> Key: HIVE-15467
> URL: https://issues.apache.org/jira/browse/HIVE-15467
> Project: Hive
> Issue Type: Bug
> Reporter: Pengcheng Xiong
> Assignee: Prasanth Jayachandran
>
> here is part of the log before it hangs
> {code}
> 2016-12-19T15:21:05,779 INFO [LlapScheduler]
> tezplugins.LlapTaskSchedulerService: ScheduleResult for Task:
> TaskInfo{task=attempt_1482189645956_0001_33_00_000000_1, priority=1,
> startTime=0, containerId=null, assignedNode=, uniqueId=54,
> localityDelayTimeout=0} = DELAYED_RESOURCES
> 2016-12-19T15:21:05,779 DEBUG [LlapScheduler]
> tezplugins.LlapTaskSchedulerService: Attempting to preempt on any host for
> task=attempt_1482189645956_0001_33_00_000000_1, pendingPreemptions=0
> 2016-12-19T15:21:05,779 INFO [LlapScheduler]
> tezplugins.LlapTaskSchedulerService: Preempting for
> task=attempt_1482189645956_0001_33_00_000000_1 on any available host
> 2016-12-19T15:21:05,779 DEBUG [LlapScheduler]
> tezplugins.LlapTaskSchedulerService: Unable to schedule all requests at
> priority=1. Skipping subsequent priority levels
> 2016-12-19T15:21:07,953 DEBUG [AMReporterQueueDrainer] impl.AMReporter:
> Removing am localhost:61788 with last associated dag
> QueryIdentifier{appIdentifier='application_1482189645956_0001',
> dagIdentifier=33} from heartbeat with taskCount=0, amFailed=false
> 2016-12-19T15:21:08,634 INFO [86edca30-bf12-42f8-90cd-a9fbdfbcb546 main]
> SessionState: Map 1: 0(+1,-1)/1
> 2016-12-19T15:21:11,700 INFO [86edca30-bf12-42f8-90cd-a9fbdfbcb546 main]
> SessionState: Map 1: 0(+1,-1)/1
> 2016-12-19T15:21:14,755 INFO [86edca30-bf12-42f8-90cd-a9fbdfbcb546 main]
> SessionState: Map 1: 0(+1,-1)/1
> 2016-12-19T15:21:17,814 INFO [86edca30-bf12-42f8-90cd-a9fbdfbcb546 main]
> SessionState: Map 1: 0(+1,-1)/1
> 2016-12-19T15:21:20,871 INFO [86edca30-bf12-42f8-90cd-a9fbdfbcb546 main]
> SessionState: Map 1: 0(+1,-1)/1
> 2016-12-19T15:21:23,931 INFO [86edca30-bf12-42f8-90cd-a9fbdfbcb546 main]
> SessionState: Map 1: 0(+1,-1)/1
> 2016-12-19T15:21:26,977 INFO [86edca30-bf12-42f8-90cd-a9fbdfbcb546 main]
> SessionState: Map 1: 0(+1,-1)/1
> 2016-12-19T15:21:30,027 INFO [86edca30-bf12-42f8-90cd-a9fbdfbcb546 main]
> SessionState: Map 1: 0(+1,-1)/1
> 2016-12-19T15:21:33,078 INFO [86edca30-bf12-42f8-90cd-a9fbdfbcb546 main]
> SessionState: Map 1: 0(+1,-1)/1
> 2016-12-19T15:21:36,133 INFO [86edca30-bf12-42f8-90cd-a9fbdfbcb546 main]
> SessionState: Map 1: 0(+1,-1)/1
> 2016-12-19T15:21:39,179 INFO [86edca30-bf12-42f8-90cd-a9fbdfbcb546 main]
> SessionState: Map 1: 0(+1,-1)/1
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
