Hi, I am using Apache Beam 2.14.0 with Flink 1.8.0 and I have included the RocksDb dependency in my projects pom.xml as well as baked it into the Dockerfile like this:
FROM flink:1.8.0-scala_2.11 ADD --chown=flink:flink http://central.maven.org/maven2/org/apache/flink/flink-statebackend-rocksdb_2.11/1.8.0/flink-statebackend-rocksdb_2.11-1.8.0.jar /opt/flink/lib/flink-statebackend-rocksdb_2.11-1.8.0.jar Everything seems to be normal up to the point when I try to stop and cleanly shutdown my pipeline. I get the following error: java.lang.NoSuchMethodError: org.rocksdb.ColumnFamilyHandle.getDescriptor()Lorg/rocksdb/ColumnFamilyDescriptor; at org.apache.flink.contrib.streaming.state.RocksDBOperationUtils.addColumnFamilyOptionsToCloseLater(RocksDBOperationUtils.java:160) at org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend.dispose(RocksDBKeyedStateBackend.java:331) at org.apache.flink.streaming.api.operators.AbstractStreamOperator.dispose(AbstractStreamOperator.java:362) at org.apache.beam.runners.flink.translation.wrappers.streaming.DoFnOperator.dispose(DoFnOperator.java:470) at org.apache.flink.streaming.runtime.tasks.StreamTask.tryDisposeAllOperators(StreamTask.java:454) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:337) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) at java.lang.Thread.run(Thread.java:748) I can cancel my pipeline and snapshotting in general works, however. Flink 1.7.2 with Beam 2.12.0 did not have any problem, could it be that this is caused by the switch to FRocksDb?[0] Best, Tobias [0] https://ci.apache.org/projects/flink/flink-docs-stable/release-notes/flink-1.8.html#rocksdb-version-bump-and-switch-to-frocksdb-flink-10471
