I agree with Piotr's analysis. It should not matter whether you are using RocksDBStateBackend or not. It seems as if you have a Hadoop dependency clash. Could you check which dependencies are on the class path?
Cheers, Till On Thu, Aug 20, 2020 at 3:52 PM Piotr Nowojski <piotr.nowoj...@gmail.com> wrote: > Hi, > > It looks more like a dependency convergence issue - you have two > conflicting versions of > `org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest` > on the class path. Or you built your jar with one version and trying to > execute it with a different one. > > Till is it some kind of a known issue? > > Piotrek > > > czw., 20 sie 2020 o 06:48 Vijayendra Yadav <contact....@gmail.com> > napisał(a): > >> Hi Team, >> >> Getting the following error when using RocksDBStateBackend on yarn/EMR. >> Am I missing any dependencies? >> >> >> 2020-08-20 04:37:00,713 ERROR >> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl - >> Exception on heartbeat >> java.lang.NoSuchMethodError: >> org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest.newInstance(IFLjava/util/List;Ljava/util/List;Ljava/util/List;Lorg/apache/hadoop/yarn/api/records/ResourceBlacklistRequest;)Lorg/apache/hadoop/yarn/api/protocolrecords/AllocateRequest; >> at >> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.allocate(AMRMClientImpl.java:280) >> at >> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$HeartbeatThread.run(AMRMClientAsyncImpl.java:273) >> 2020-08-20 04:37:00,714 INFO >> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl - >> Interrupted while waiting for queue >> java.lang.InterruptedException >> at >> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) >> at >> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) >> at >> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) >> at >> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:323) >> 2020-08-20 04:37:00,714 ERROR >> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl - Stopping >> callback due to: >> java.lang.NoSuchMethodError: >> org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest.newInstance(IFLjava/util/List;Ljava/util/List;Ljava/util/List;Lorg/apache/hadoop/yarn/api/records/ResourceBlacklistRequest;)Lorg/apache/hadoop/yarn/api/protocolrecords/AllocateRequest; >> at >> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.allocate(AMRMClientImpl.java:280) >> at >> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$HeartbeatThread.run(AMRMClientAsyncImpl.java:273) >> 2020-08-20 04:37:00,714 ERROR org.apache.flink.yarn.YarnResourceManager >> - Fatal error occurred in ResourceManager. >> java.lang.NoSuchMethodError: >> org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest.newInstance(IFLjava/util/List;Ljava/util/List;Ljava/util/List;Lorg/apache/hadoop/yarn/api/records/ResourceBlacklistRequest;)Lorg/apache/hadoop/yarn/api/protocolrecords/AllocateRequest; >> at >> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.allocate(AMRMClientImpl.java:280) >> at >> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$HeartbeatThread.run(AMRMClientAsyncImpl.java:273) >> 2020-08-20 04:37:00,714 ERROR >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Fatal error >> occurred in the cluster entrypoint. >> java.lang.NoSuchMethodError: >> org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest.newInstance(IFLjava/util/List;Ljava/util/List;Ljava/util/List;Lorg/apache/hadoop/yarn/api/records/ResourceBlacklistRequest;)Lorg/apache/hadoop/yarn/api/protocolrecords/AllocateRequest; >> at >> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.allocate(AMRMClientImpl.java:280) >> at >> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$HeartbeatThread.run(AMRMClientAsyncImpl.java:273) >> 2020-08-20 04:37:00,718 INFO org.apache.flink.runtime.blob.BlobServer >> >> - Stopped BLOB server at 0.0.0.0:45627 >> >> >> Regards, >> >> Vijay >> >>
