Great to hear that you fixed the problem!
Cheers,
Till
On Mon, Aug 24, 2020 at 2:53 AM Vijayendra Yadav
wrote:
> Thank You Till. I had an old hadoop version dependency in one of the
> dependent jars causing conflict.
>
> On Fri, Aug 21, 2020 at 12:24 AM Till Rohrmann
> wrote:
>
>> Hi Vijay,
>
Thank You Till. I had an old hadoop version dependency in one of the
dependent jars causing conflict.
On Fri, Aug 21, 2020 at 12:24 AM Till Rohrmann wrote:
> Hi Vijay,
>
> could you move the s3 filesystem
> dependency lib/flink-s3-fs-hadoop-1.10.0.jar into the plugin directory? See
> this link
Hi Vijay,
could you move the s3 filesystem
dependency lib/flink-s3-fs-hadoop-1.10.0.jar into the plugin directory? See
this link [1] for more information. Since Flink 1.10 we have removed the
relocation of filesystem dependencies because the recommended way to load
them is via Flink's plugin mecha
Hi Till/ Piotr,
*My process was working with : FsStateBackend but when I switched
to RocksDBStateBackend I faced this problem. My class path is below. *
*Related jar in classpath: *
/usr/lib/hadoop-yarn/hadoop-yarn-api-2.8.5-amzn-6.jar:/usr/lib/hadoop-yarn/hadoop-yarn-api.jar:
*Classpath:*
:co
Here is a link to information on how to integrate Flink with Hadoop [1]. In
the latest version you only need to point Flink to the Hadoop libraries via
setting the HADOOP_CLASSPATH environment variable.
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/hadoop.html
Ch
I agree with Piotr's analysis. It should not matter whether you are using
RocksDBStateBackend or not. It seems as if you have a Hadoop dependency
clash. Could you check which dependencies are on the class path?
Cheers,
Till
On Thu, Aug 20, 2020 at 3:52 PM Piotr Nowojski
wrote:
> Hi,
>
> It look
Hi,
It looks more like a dependency convergence issue - you have two
conflicting versions of
`org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest`
on the class path. Or you built your jar with one version and trying to
execute it with a different one.
Till is it some kind of a known issue?
Hi Team,
Getting the following error when using RocksDBStateBackend on yarn/EMR. Am
I missing any dependencies?
2020-08-20 04:37:00,713 ERROR
org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl -
Exception on heartbeat
java.lang.NoSuchMethodError:
org.apache.hadoop.yarn.api.protoco
