Hi yun: I didn't try to build rocksdb with vagrant, but just `make -j8 rocksdbjava` directly in an ARM machine. We hit some issues as well. My colleague has created an issue in rocksdb[1]. Rocksdb doesn't contains ARM .so file in his offical jar package. If you have the same request, let's work together there.
Thanks. [1]: https://github.com/facebook/rocksdb/issues/5559 Yun Tang <myas...@live.com> 于2019年7月11日周四 下午12:01写道： > Hi Xiyuan > > Have you ever tried to release RocksDB on ARM like official doc[1] > suggests? From our experience, cross-building for ARM did not work fine > with Vagrant and we have to build rocksDB's binary file on ARM separately. > > As frocksdb [2] might not always maintained in Flink, I think we'd better > support to release RocksDB-java with ARM officially. > > > [1] https://github.com/facebook/rocksdb/blob/master/java/RELEASE.md > [2] https://github.com/dataArtisans/frocksdb > > Best > Yun Tang > > > ________________________________ > From: Xiyuan Wang <wangxiyuan1...@gmail.com> > Sent: Tuesday, July 9, 2019 10:52 > To: dev@flink.apache.org > Subject: Re: [DISCUSS] ARM support for Flink > > Thanks for your help. I built the frocksdb locally on ARM and all the > related tests are passed now. Except some tests which can be fixed easily, > it seems that both building and testing are ran well on ARM. > > Basing on my test, Is it possible to support Flink on ARM officailly? Seem > the worklist is not too long. And I can help with the CI testing part. > > Need Flink team's idea. > > Thanks. > > Dian Fu <dian0511...@gmail.com> 于2019年7月8日周一 上午10:23写道： > > > Hi Xiyuan, > > > > Thanks for bring the discussion. > > > > WRT the exception, it's because the native bundled in the rocksdb jar > file > > isn't compiled with cross platform support. You can refer [1] for how to > > build rocksdb which has ARM platform. > > > > WRT ARM support, the rocksdb currently used in Flink is hosted in the > > Ververica git [2], so it won't be difficult to make it support ARM. > > However, I guess this git exists just for temporary [3], not because we > > want to add much feature in rocksdb. > > > > [1] https://github.com/facebook/rocksdb/issues/678 < > > https://github.com/facebook/rocksdb/issues/678> > > [2] https://github.com/dataArtisans/frocksdb < > > https://github.com/dataArtisans/frocksdb> > > [3] https://issues.apache.org/jira/browse/FLINK-10471 < > > https://issues.apache.org/jira/browse/FLINK-10471> > > > > Regards, > > Dian > > > > > 在 2019年7月8日，上午9:17，Xiyuan Wang <wangxiyuan1...@gmail.com> 写道： > > > > > > Hi Flink: > > > Recently we meet a problem that we have to test and run Flink on ARM > > > arch. While after searching Flink community, I didn’t find an official > > ARM > > > release version. > > > > > > Since Flink is made by Java and Scala language which can be ran > > > cross-platform usually, I think Flink can be built and ran on ARM > > directly > > > as well. Then with my local test, Flink was built and deployed success > as > > > expected. But some tests were failed due to ARM arch. For example: > > > > > > 1. MemoryArchitectureTest.testArchitectureNotUnknown:34 Values should > be > > > different. Actual: UNKNOWN > > > 2. [ERROR] > > > > > > testIterator(org.apache.flink.contrib.streaming.state.RocksDBRocksStateKeysIteratorTest) > > > Time elapsed: 0.234 s <<< ERROR! > > > java.io.IOException: Could not load the native RocksDB library > > > at > > > > > > org.apache.flink.contrib.streaming.state.RocksDBRocksStateKeysIteratorTest.testIteratorHelper(RocksDBRocksStateKeysIteratorTest.java:90) > > > at > > > > > > org.apache.flink.contrib.streaming.state.RocksDBRocksStateKeysIteratorTest.testIterator(RocksDBRocksStateKeysIteratorTest.java:63) > > > Caused by: java.lang.UnsatisfiedLinkError: > > > > > > /tmp/rocksdb-lib-81ca7930b92af2cca143a050c0338d34/librocksdbjni-linux64.so: > > > > > > /tmp/rocksdb-lib-81ca7930b92af2cca143a050c0338d34/librocksdbjni-linux64.so: > > > cannot open shared object file: No such file or directory (Possible > > cause: > > > can't load AMD 64-bit .so on a AARCH64-bit platform) > > > … > > > > > > Since the test isn’t passed totally, we are not sure if Flink 100% > > > support ARM or not. Is it possible for Flink to support ARM release > > > officially? I guess it may be not a very huge work basing on Java. I > > notice > > > that Flink now uses trivis-ci which is X86 only for build & test check. > > Is > > > it possible to add an ARM arch CI as well? It can be non-voting first. > > Then > > > we can keep monitoring and fixing ARM related error. One day it’s > stable > > > enough, we can remove the non-voting tag and create Flink ARM release. > > > > > > There is an open source CI community called OpenLab[1] which can > provide > > > CI function and ARM resource to Flink by free. I’m one of the OpenLab > > > members. If Flink commun think ARM support is fine, I can keep helping > > > Flink to build and maintain the ARM CI job. There is an POC for Flink > > ARM > > > build job made by me on OpenLab system[2] and a live demo which built > and > > > run on an ARM VM[3]. You can take a look first. > > > > > > Eager to get everybody’s feedback. Any question is welcome. > > > > > > Thanks. > > > > > > [1]: https://openlabtesting.org/ > > > [2]: https://github.com/theopenlab/flink/pull/1 > > > [3]: http://114.115.168.52:8081/#/overview > > > > >
