Sergey Shelukhin created HIVE-12199:
---------------------------------------
Summary: LLAP: unnecessary preemption of the same stage
Key: HIVE-12199
URL: https://issues.apache.org/jira/browse/HIVE-12199
Project: Hive
Issue Type: Bug
Reporter: Sergey Shelukhin
6-node cluster x 16 executors, nothing else running on it
{code}
hive> select sum(l_extendedprice * l_discount) as revenue from testing.lineitem
where l_shipdate >= '1993-01-01' and l_shipdate < '1994-01-01' ;
...
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING
FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... llap SUCCEEDED 176 176 0 0
0 8
Reducer 2 ...... llap SUCCEEDED 1 1 0 0
0 1
{code}
those killed mappers are preempted:
{code}
015-10-15 18:07:07,154 INFO [Dispatcher thread: Central]
history.HistoryEventHandler:
[HISTORY][DAG:dag_1442254312093_2158_1][Event:TASK_ATTEMPT_FINISHED]:
vertexName=Map 1, taskAttemptId=attempt_1442254312093_2158_1_00_000103_0,
startTime=1444946823795, finishTime=1444946827152, timeTaken=3357,
status=KILLED, errorEnum=EXTERNAL_PREEMPTION, diagnostics=Attempt preempted,
lastDataEventSourceTA=null, lastDataEventTime=0, counters=Counters: 1,
org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1
{code}
There's no reason for mappers of the same stage to preempt each other in this
case
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
