[
https://issues.apache.org/jira/browse/IGNITE-2343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15091916#comment-15091916
]
Vladimir Ozerov commented on IGNITE-2343:
-----------------------------------------
It seems that the root cause is identified. Splunk/Hunk has stateful InputSplit
and expects that it will be created only once. To the contrast, on the map step
Ignite could create the InputSplit twice:
1) HadoopV2MapTask.run0;
2) And then in HadoopV2Context.getInputSplit().
Splunk/Hunk initializes the first instance, then we drop it, and then pass the
second unitialized instance. Result - NPE.
Proposed fix: keep the first initialized instance. To achieve this we may
always call HadoopV2Context.getInputSplit(). But two things must be cleared:
1) What to do in case null is returned.
2) Do not convert IgniteCheckedException to IllegalStateException when calling
from HadoopV2MapTask.run0().
> IGFS: MapReduce task sent by Hunk/Splunk fails with NPE
> -------------------------------------------------------
>
> Key: IGNITE-2343
> URL: https://issues.apache.org/jira/browse/IGNITE-2343
> Project: Ignite
> Issue Type: Bug
> Components: IGFS
> Affects Versions: ignite-1.4
> Reporter: Denis Magda
> Assignee: Vladimir Ozerov
> Labels: important
> Fix For: 1.6
>
>
> A MR task being sent by Hunk [1] fails with NPE
> {noformat}
> [10:31:58,250][ERROR][Hadoop-task-fd3ff783-d952-4147-8e5b-8e3080b91ddb_1-MAP-50-
> 0-#271%null%][HadoopRunnableTask] Task execution failed.
> class org.apache.ignite.IgniteCheckedException: class
> org.apache.ignite.IgniteCh eckedException: null
> at org.apache.ignite.internal.processors.hadoop.v2.HadoopV2MapTask.run0(
> HadoopV2MapTask.java:105)
> at org.apache.ignite.internal.processors.hadoop.v2.HadoopV2Task.run(Hado
> opV2Task.java:54)
> at org.apache.ignite.internal.processors.hadoop.v2.HadoopV2TaskContext.r
> un(HadoopV2TaskContext.java:249)
> at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunna
> bleTask.runTask(HadoopRunnableTask.java:201)
> at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunna
> bleTask.call0(HadoopRunnableTask.java:144)
> at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunna
> bleTask$1.call(HadoopRunnableTask.java:116)
> at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunna
> bleTask$1.call(HadoopRunnableTask.java:114)
> at org.apache.ignite.internal.processors.hadoop.v2.HadoopV2TaskContext$1
> .run(HadoopV2TaskContext.java:550)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma
> tion.java:1657)
> at org.apache.ignite.internal.processors.hadoop.v2.HadoopV2TaskContext.r
> unAsJobOwner(HadoopV2TaskContext.java:548)
> at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunna
> bleTask.call(HadoopRunnableTask.java:114)
> at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunna
> bleTask.call(HadoopRunnableTask.java:46)
> at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopExecu
> torService$2.body(HadoopExecutorService.java:186)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java :110)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at com.splunk.mr.SplunkSearchMapper.runImpl(SplunkSearchMapper.java:308)
> at com.splunk.mr.SplunkBaseMapper.run(SplunkBaseMapper.java:169)
> at org.apache.ignite.internal.processors.hadoop.v2.HadoopV2MapTask.run0(
> HadoopV2MapTask.java:87)
> ... 16 more
> at org.apache.ignite.internal.processors.hadoop.HadoopUtils.transformExc
> eption(HadoopUtils.java:290)
> at org.apache.ignite.internal.processors.hadoop.v2.HadoopV2TaskContext.r
> un(HadoopV2TaskContext.java:255)
> at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunna
> bleTask.runTask(HadoopRunnableTask.java:201)
> at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunna
> bleTask.call0(HadoopRunnableTask.java:144)
> at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunna
> bleTask$1.call(HadoopRunnableTask.java:116)
> at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunna
> bleTask$1.call(HadoopRunnableTask.java:114)
> at org.apache.ignite.internal.processors.hadoop.v2.HadoopV2TaskContext$1
> .run(HadoopV2TaskContext.java:550)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma
> tion.java:1657)
> at org.apache.ignite.internal.processors.hadoop.v2.HadoopV2TaskContext.r
> unAsJobOwner(HadoopV2TaskContext.java:548)
> at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunna
> bleTask.call(HadoopRunnableTask.java:114)
> at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunna
> bleTask.call(HadoopRunnableTask.java:46)
> at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopExecu
> torService$2.body(HadoopExecutorService.java:186)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java :110)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> [1] http://www.splunk.com/en_us/products/hunk.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
