Hi all,
Recently Travis announced that ARM arch is in Alpha release[1]. Since Flink
has integrated with Travis already, I think it's quite easy for Flink to
use it for ARM CI.
Maybe some of you know that I'm working on Flink ARM testing and support. I
suggested to use OpenLab[2] as the ARM CI infrastructure before. Though
it's not hard to use OpenLab, it'll still introduce some new concept or
burden to Flink. Flink team has another choice now.
And as the discussion before, we can still run ARM CI as Cron job first. I
have ran POC e2e test in OpenLab for some days[3](Of cause, it can be
changed to Travis).
Following travis x86 test, it includes:
flink-end-to-end-test-part1
split_checkpoints.sh and split_sticky.sh
flink-end-to-end-test-part2
split_heavy.sh and split_ha.sh
flink-end-to-end-test-part3
split_misc.sh and split_misc_hadoopfree.sh
part1 and part2 runs well. part3 is not statble. I need take more time to
fix part3. container part is not included because the problem5 mentioned
below.
While I did som hacks to make sure the job pass. It includes:
1. Frocksdb ARM package: https://issues.apache.org/jira/browse/FLINK-13598
(Not solved)
2. PrometheusReporterEndToEndITCase doesn't support ARM arch:
https://issues.apache.org/jira/browse/FLINK-14086 (PR for fix:
https://github.com/apache/flink/pull/9768)
3. Elasticsearch Xpack Machine Learning doesn't support ARM :
https://issues.apache.org/jira/browse/FLINK-14126 (PR for fix:
https://github.com/apache/flink/pull/9765)
4. maven-shade-plugin 3.2.1 doesn't work on ARM for Flink (Fixed, thanks
@Dian Fu )
5. flink e2e container test doesn't support ARM:
https://issues.apache.org/jira/browse/FLINK-14241 (PR for fix:
https://github.com/apache/flink/pull/9782)
No matter which CI Flink will use, all the bug mentioned above should be
fixed. Please help review these PRs. And if you have any question, please
let me know.
No matter which CI Flink will choose, I'd like to keep working on Flink ARM
support and keep testing and fixing ARM related bugs.
Thanks very much.
[1]: https://blog.travis-ci.com/2019-10-07-multi-cpu-architecture-support
[2]: https://openlabtesting.org/
[3]: http://status.openlabtesting.org/builds?project=apache%2Fflink
