FYI Maybe this is an env problem. I encountered this problem when running flink 1.9 on k8s, but it was success when running on yarn. did not figure out why this happened, will update here after find it out.
Best, Congxian Thad Truman <ttru...@neovest.com> 于2019年10月23日周三 上午1:33写道： > Hi Samya, > > > > Were you able to get this resolved? Seeing the same issue here after > upgrading to Flink 1.9 from 1.6. > > > > Thanks, > > > > Thad > > > > *From:* Andrey Zagrebin <and...@ververica.com> > *Sent:* Wednesday, July 3, 2019 9:09 AM > *To:* Haibo Sun <sunhaib...@163.com> > *Cc:* Patro, Samya <samya.pa...@gs.com>; user@flink.apache.org; Bari, > Swapnil <swapnil.b...@gs.com> > *Subject:* Re: Could not load the native RocksDB library > > > > Hi Samya, > > > > Additionally to Haibo's answer: > > Have you tried the previous 1.7 version of Flink? The Rocksdb version was > upgraded in 1.8 version. > > > > Best, > > Andrey > > > > On Wed, Jul 3, 2019 at 5:21 AM Haibo Sun <sunhaib...@163.com> wrote: > > Hi, Samya.Patro > > > > I guess this may be a setup problem. What OS and what version of JDK do > you use? You can try upgrading JDK to see if the issue can be solved. > > > > Best, > > Haibo > > > At 2019-07-02 17:16:59, "Patro, Samya" <samya.pa...@gs.com> wrote: > > Hello, > I am using rocksdb for storing state . But when I run the pipeline I get > the error ”Could not load the native RocksDB library” . Kindly can you > check the configs and error stacktrace and suggest what am I doing wrong . > > > > Flink version - 1.8.0 > > > > <*dependency*> > <*groupId*>org.apache.flink</*groupId*> > <*artifactId*>flink-statebackend-rocksdb_2.11</*artifactId*> > <*version*>1.8.0</*version*> > </*dependency*> > > > > This is the flink checkpointing config I have used > > > > *executionEnvironment**.enableCheckpointing(**300000* > > *); > executionEnvironment.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.AT_LEAST_ONCE); > executionEnvironment.getCheckpointConfig().setMinPauseBetweenCheckpoints(* > *50000**);* > > *executionEnvironment.getCheckpointConfig().setCheckpointTimeout(**600000* > *); > executionEnvironment.getCheckpointConfig().setMaxConcurrentCheckpoints(* > *1**);* > > *executionEnvironment* > *.getCheckpointConfig().enableExternalizedCheckpoints(CheckpointConfig.ExternalizedCheckpointCleanup.* > *RETAIN_ON_CANCELLATION**);* > > *StateBackend rocksDbBackend = **new **RocksDBStateBackend(parameter.get(* > *"stateBackendPath"**),**true* > *); **executionEnvironment**.setStateBackend(rocksDbBackend);* > > > > *When I run the pipeline, I get this error* > > > > java.lang.Exception: Exception while creating StreamOperatorStateContext. > > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:195) > > at > org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:250) > > at > org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:738) > > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:289) > > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) > > at java.lang.Thread.run(Thread.java:745) > > Caused by: org.apache.flink.util.FlinkException: Could not restore keyed > state backend for StreamFlatMap_9dd63673dd41ea021b896d5203f3ba7c_(1/5) from > any of the 1 provided restore options. > > at > org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135) > > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:307) > > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:135) > > ... 5 more > > Caused by: java.io.IOException: Could not load the native RocksDB library > > at > org.apache.flink.contrib.streaming.state.RocksDBStateBackend.ensureRocksDBIsLoaded(RocksDBStateBackend.java:911) > > at > org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:482) > > at > org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:291) > > at > org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:142) > > at > org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:121) > > ... 7 more > > Caused by: java.lang.UnsatisfiedLinkError: > /tmp/rocksdb-lib-ee961b2f013f7d5baabbc1cb2b0b87d7/librocksdbjni-linux64.so: > /tmp/rocksdb-lib-ee961b2f013f7d5baabbc1cb2b0b87d7/librocksdbjni-linux64.so: > undefined symbol: malloc_stats_print > > at java.lang.ClassLoader$NativeLibrary.load(Native Method) > > at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) > > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) > > at java.lang.Runtime.load0(Runtime.java:809) > > at java.lang.System.load(System.java:1086) > > at > org.rocksdb.NativeLibraryLoader.loadLibraryFromJar(NativeLibraryLoader.java:78) > > at > org.rocksdb.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:56) > > at > org.apache.flink.contrib.streaming.state.RocksDBStateBackend.ensureRocksDBIsLoaded(RocksDBStateBackend.java:888) > > ... 11 more > > > > > > > > *Thanks and Regards, Samya Ranjan Patro Goldman sachs* > > > > > ------------------------------ > > > Your Personal Data: We may collect and process information about you that > may be subject to data protection laws. For more information about how we > use and disclose your personal data, how we protect your information, our > legal basis to use your information, your rights and who you can contact, > please refer to: www.gs.com/privacy-notices > >
