Hi all, I am trying to enable tez auxiliary service tez shuffler. >From hadoop logs it's confirmed that tez_shuffle is running
> INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: > Adding auxiliary service tez_shuffle, "tez_shuffle" > INFO org.apache.tez.auxservices.ShuffleHandler: tez_shuffle listening on > port 13563 > tez-site.xml ------------- ... <property> <name>tez.am.shuffle.auxiliary-service.id</name> <value>tez_shuffle</value> </property> yarn-site.xml ------------- ... <property> <name>yarn.nodemanager.aux-services</name> <value>tez_shuffle</value> </property> <property> <name>yarn.nodemanager.aux-services.tez_shuffle.class</name> <value>org.apache.tez.auxservices.ShuffleHandler</value> </property> * On hive-llap I am getting below error:-* TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) : attempt_1594912627497_0009_1_01_000002_3:java.lang.RuntimeException: java.lang.RuntimeException: Map operator initialization failed at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:300) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:251) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: Map operator initialization failed at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:363) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267) ... 15 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Async Initialization failed. abortRequested=false at org.apache.hadoop.hive.ql.exec.Operator.completeInitialization(Operator.java:466) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:400) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:573) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:525) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:386) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:335) ... 16 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.tez.LlapObjectCache.retrieve(LlapObjectCache.java:118) at org.apache.hadoop.hive.ql.exec.tez.LlapObjectCache$1.call(LlapObjectCache.java:143) ... 4 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.tez.HashTableLoader.load(HashTableLoader.java:280) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTableInternal(MapJoinOperator.java:327) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:396) at org.apache.hadoop.hive.ql.exec.MapJoinOperator.lambda$initializeOp$0(MapJoinOperator.java:203) at org.apache.hadoop.hive.ql.exec.tez.LlapObjectCache.retrieve(LlapObjectCache.java:116) ... 5 more *Caused by: java.lang.NullPointerException at org.apache.tez.runtime.api.impl.TezTaskContextImpl.getServiceConsumerMetaData(TezTaskContextImpl.java:190) at org.apache.tez.runtime.library.common.shuffle.impl.ShuffleManager.<init>(ShuffleManager.java:264)* at org.apache.tez.runtime.library.input.UnorderedKVInput.start(UnorderedKVInput.java:146) at org.apache.hadoop.hive.ql.exec.tez.HashTableLoader.load(HashTableLoader.java:277) ... 9 more Regards, Manoj Kumar
