+1
And I want to second Arvid's mention of testcontainers [1].
[1] https://www.testcontainers.org/
On 18.11.20 10:43, Yang Wang wrote:
Thanks till and Jark for sharing the information.
I am also +1 for this proposal and glad to wire the new introduced K8s HA
e2e tests to java based framework.
Best,
Yang
Jark Wu <imj...@gmail.com> 于2020年11月18日周三 下午5:23写道：
+1 to use the Java-based testing framework and +1 for using docker images
in the future.
IIUC, the Java-based testing framework refers to the
`flink-end-to-end-tests-common` module.
The java-based framework helped us a lot when debugging the unstable e2e
tests.
Best,
Jark
On Wed, 18 Nov 2020 at 14:42, Yang Wang <danrtsey...@gmail.com> wrote:
Thanks for starting this discussion.
In general, I agree with you that a java-based testing framework is
better
than the bash-based. It will
help a lot for the commons and utilities.
Since I am trying to add a new bash-based Kubernetes HA test, I have some
quick questions.
* I am not sure where the java-based framework is. Do you mean
"flink-jepsen" module or sth else?
* Maybe it will be harder to run a cli command(e.g. flink run /
run-application) to submit a Flink job in the java-based framework.
* It will be harder to inject some operations. For example, kill the
JobManager in Kubernetes. Currently, I
am trying to use "kubectl exec" to do this.
Best,
Yang
Robert Metzger <rmetz...@apache.org> 于2020年11月17日周二 下午11:36写道：
Hi all,
Since we are currently testing the 1.12 release, and potentially adding
more automated e2e tests, I would like to bring up our end to end tests
for
discussion.
Some time ago, we introduced a Java-based testing framework, with the
idea
of replacing the current bash-based end to end tests.
Since the introduction of the java-based framework, more bash tests
were
actually added, making a future migration even harder.
*For that reason, I would like to propose that we are stopping to add
any
new bash end to end tests to Flink. All new end to end tests must be
written in Java and rely on the existing testing framework.*
For the 1.13 release, I'm trying to find some time to revisit potential
improvements for the existing java e2e framework (such as using Docker
images everywhere), as well as a migration plan for the existing bash
tests. We have a large number of bash e2e tests that are just
parameterized
differently. If we would start migrating them to Java, we could move a
larger proportion of tests over to the new Java framework, and tackle
the
more involved bash tests later (kerberized yarn, kubernetes, ...).
Let me know what you think!
Best,
Robert
PS: If you are wondering why I'm bringing this up now: I'm spending
quite a
lot of time trying to figure out really hard to debug issues with our
bash
testing infra.
Also, it is very difficult to introduce something generic for all tests
(such as a test-timeout, using docker as the preferred deployment
method
etc.) since the tests often don't share common tooling.
Speaking about tooling: there are a lot of utilities everywhere,
sometimes
duplicated, with different features / stability etc.
I believe bash is not the right tool for a project this size (in terms
of
developers and lines of code)
