Hi Xiyuan, Thanks for bring the discussion.
WRT the exception, it's because the native bundled in the rocksdb jar file isn't compiled with cross platform support. You can refer [1] for how to build rocksdb which has ARM platform. WRT ARM support, the rocksdb currently used in Flink is hosted in the Ververica git [2], so it won't be difficult to make it support ARM. However, I guess this git exists just for temporary [3], not because we want to add much feature in rocksdb. [1] https://github.com/facebook/rocksdb/issues/678 <https://github.com/facebook/rocksdb/issues/678> [2] https://github.com/dataArtisans/frocksdb <https://github.com/dataArtisans/frocksdb> [3] https://issues.apache.org/jira/browse/FLINK-10471 <https://issues.apache.org/jira/browse/FLINK-10471> Regards, Dian > 在 2019年7月8日，上午9:17，Xiyuan Wang <wangxiyuan1...@gmail.com> 写道： > > Hi Flink: > Recently we meet a problem that we have to test and run Flink on ARM > arch. While after searching Flink community, I didn’t find an official ARM > release version. > > Since Flink is made by Java and Scala language which can be ran > cross-platform usually, I think Flink can be built and ran on ARM directly > as well. Then with my local test, Flink was built and deployed success as > expected. But some tests were failed due to ARM arch. For example: > > 1. MemoryArchitectureTest.testArchitectureNotUnknown:34 Values should be > different. Actual: UNKNOWN > 2. [ERROR] > testIterator(org.apache.flink.contrib.streaming.state.RocksDBRocksStateKeysIteratorTest) > Time elapsed: 0.234 s <<< ERROR! > java.io.IOException: Could not load the native RocksDB library > at > org.apache.flink.contrib.streaming.state.RocksDBRocksStateKeysIteratorTest.testIteratorHelper(RocksDBRocksStateKeysIteratorTest.java:90) > at > org.apache.flink.contrib.streaming.state.RocksDBRocksStateKeysIteratorTest.testIterator(RocksDBRocksStateKeysIteratorTest.java:63) > Caused by: java.lang.UnsatisfiedLinkError: > /tmp/rocksdb-lib-81ca7930b92af2cca143a050c0338d34/librocksdbjni-linux64.so: > /tmp/rocksdb-lib-81ca7930b92af2cca143a050c0338d34/librocksdbjni-linux64.so: > cannot open shared object file: No such file or directory (Possible cause: > can't load AMD 64-bit .so on a AARCH64-bit platform) > … > > Since the test isn’t passed totally, we are not sure if Flink 100% > support ARM or not. Is it possible for Flink to support ARM release > officially? I guess it may be not a very huge work basing on Java. I notice > that Flink now uses trivis-ci which is X86 only for build & test check. Is > it possible to add an ARM arch CI as well? It can be non-voting first. Then > we can keep monitoring and fixing ARM related error. One day it’s stable > enough, we can remove the non-voting tag and create Flink ARM release. > > There is an open source CI community called OpenLab[1] which can provide > CI function and ARM resource to Flink by free. I’m one of the OpenLab > members. If Flink commun think ARM support is fine, I can keep helping > Flink to build and maintain the ARM CI job. There is an POC for Flink ARM > build job made by me on OpenLab system[2] and a live demo which built and > run on an ARM VM[3]. You can take a look first. > > Eager to get everybody’s feedback. Any question is welcome. > > Thanks. > > [1]: https://openlabtesting.org/ > [2]: https://github.com/theopenlab/flink/pull/1 > [3]: http://114.115.168.52:8081/#/overview
