Hi Samya, Additionally to Haibo's answer: Have you tried the previous 1.7 version of Flink? The Rocksdb version was upgraded in 1.8 version.
Best, Andrey On Wed, Jul 3, 2019 at 5:21 AM Haibo Sun <sunhaib...@163.com> wrote: > Hi, Samya.Patro > > I guess this may be a setup problem. What OS and what version of JDK do > you use? You can try upgrading JDK to see if the issue can be solved. > > Best, > Haibo > > At 2019-07-02 17:16:59, "Patro, Samya" <samya.pa...@gs.com> wrote: > > Hello, > I am using rocksdb for storing state . But when I run the pipeline I get > the error ”Could not load the native RocksDB library” . Kindly can you > check the configs and error stacktrace and suggest what am I doing wrong . > > > > Flink version - 1.8.0 > > > > <*dependency*> > <*groupId*>org.apache.flink</*groupId*> > <*artifactId*>flink-statebackend-rocksdb_2.11</*artifactId*> > <*version*>1.8.0</*version*> > </*dependency*> > > > > This is the flink checkpointing config I have used > > > > *executionEnvironment**.enableCheckpointing(**300000* > > *); > executionEnvironment.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.AT_LEAST_ONCE); > executionEnvironment.getCheckpointConfig().setMinPauseBetweenCheckpoints(* > *50000**);* > > *executionEnvironment.getCheckpointConfig().setCheckpointTimeout(**600000* > *); > executionEnvironment.getCheckpointConfig().setMaxConcurrentCheckpoints(* > *1**);* > > *executionEnvironment* > *.getCheckpointConfig().enableExternalizedCheckpoints(CheckpointConfig.ExternalizedCheckpointCleanup.* > *RETAIN_ON_CANCELLATION**);* > > *StateBackend rocksDbBackend = **new **RocksDBStateBackend(parameter.get(* > *"stateBackendPath"**),**true* > *); **executionEnvironment**.setStateBackend(rocksDbBackend);* > > > > *When I run the pipeline, I get this error* > > > > java.lang.Exception: Exception while creating StreamOperatorStateContext. > > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:195) > > at > org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:250) > > at > org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:738) > > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:289) > > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) > > at java.lang.Thread.run(Thread.java:745) > > Caused by: org.apache.flink.util.FlinkException: Could not restore keyed > state backend for StreamFlatMap_9dd63673dd41ea021b896d5203f3ba7c_(1/5) from > any of the 1 provided restore options. > > at > org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135) > > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:307) > > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:135) > > ... 5 more > > Caused by: java.io.IOException: Could not load the native RocksDB library > > at > org.apache.flink.contrib.streaming.state.RocksDBStateBackend.ensureRocksDBIsLoaded(RocksDBStateBackend.java:911) > > at > org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:482) > > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:291) > > at > org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:142) > > at > org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:121) > > ... 7 more > > Caused by: java.lang.UnsatisfiedLinkError: > /tmp/rocksdb-lib-ee961b2f013f7d5baabbc1cb2b0b87d7/librocksdbjni-linux64.so: > /tmp/rocksdb-lib-ee961b2f013f7d5baabbc1cb2b0b87d7/librocksdbjni-linux64.so: > undefined symbol: malloc_stats_print > > at java.lang.ClassLoader$NativeLibrary.load(Native Method) > > at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) > > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) > > at java.lang.Runtime.load0(Runtime.java:809) > > at java.lang.System.load(System.java:1086) > > at > org.rocksdb.NativeLibraryLoader.loadLibraryFromJar(NativeLibraryLoader.java:78) > > at > org.rocksdb.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:56) > > at > org.apache.flink.contrib.streaming.state.RocksDBStateBackend.ensureRocksDBIsLoaded(RocksDBStateBackend.java:888) > > ... 11 more > > > > > > > > *Thanks and Regards, Samya Ranjan Patro Goldman sachs* > > > > ------------------------------ > > Your Personal Data: We may collect and process information about you that > may be subject to data protection laws. For more information about how we > use and disclose your personal data, how we protect your information, our > legal basis to use your information, your rights and who you can contact, > please refer to: www.gs.com/privacy-notices > >
