Hi all, Since we are currently testing the 1.12 release, and potentially adding more automated e2e tests, I would like to bring up our end to end tests for discussion.
Some time ago, we introduced a Java-based testing framework, with the idea of replacing the current bash-based end to end tests. Since the introduction of the java-based framework, more bash tests were actually added, making a future migration even harder. *For that reason, I would like to propose that we are stopping to add any new bash end to end tests to Flink. All new end to end tests must be written in Java and rely on the existing testing framework.* For the 1.13 release, I'm trying to find some time to revisit potential improvements for the existing java e2e framework (such as using Docker images everywhere), as well as a migration plan for the existing bash tests. We have a large number of bash e2e tests that are just parameterized differently. If we would start migrating them to Java, we could move a larger proportion of tests over to the new Java framework, and tackle the more involved bash tests later (kerberized yarn, kubernetes, ...). Let me know what you think! Best, Robert PS: If you are wondering why I'm bringing this up now: I'm spending quite a lot of time trying to figure out really hard to debug issues with our bash testing infra. Also, it is very difficult to introduce something generic for all tests (such as a test-timeout, using docker as the preferred deployment method etc.) since the tests often don't share common tooling. Speaking about tooling: there are a lot of utilities everywhere, sometimes duplicated, with different features / stability etc. I believe bash is not the right tool for a project this size (in terms of developers and lines of code)
