Thanks for the response Dawid. In this case I don't see a reason for
timeouts in JUnit. I agree with the previously mentioned points and I think
it doesn't make sense to use them in order to limit test duration
Piotrek
wt., 4 maj 2021 o 17:44 Dawid Wysakowicz
napisał(a):
> Hey all,
>
> Sorry I'
Hey all,
Sorry I've not been active in the thread. I think the conclusion is
rather positive and we do want to depend more on the Azure watchdog and
discourage timeouts in JUnit tests from now on. I'll update our coding
guidelines accordingly.
@Piotr Yes, this was a problem in the past, but that
Hi,
I'm ok with removing most or even all timeouts. Just one thing.
Reason behind using junit timeouts that I've heard (and I was adhering to
it) was that maven watchdog was doing the thread dump and killing the test
process using timeout based on logs inactivity. Some tests were by nature
prone
Yes, you can click on the test job where a test failed. Then you can click
on 1 artifact produced (or on the overview page on the X published
(artifacts)). This brings you to the published artifacts page where we
upload for every job the logs.
Cheers,
Till
On Fri, Apr 30, 2021 at 9:22 AM Dong Lin
Thanks Till. Yes you are right. The INFO logging is enabled. It is just
dumped to a file (the FileAppender) other than the console.
There is probably a way to retrieve that log file from AZP. I will ask
other colleagues how to get this later.
On Thu, Apr 29, 2021 at 4:51 PM Till Rohrmann wrote:
I think for the maven tests we use this log4j.properties file [1].
[1] https://github.com/apache/flink/blob/master/tools/ci/log4j.properties
Cheers,
Till
On Wed, Apr 28, 2021 at 4:47 AM Dong Lin wrote:
> Thanks for the detailed explanations! Regarding the usage of timeout, now I
> agree that i
Thanks for the detailed explanations! Regarding the usage of timeout, now I
agree that it is better to remove per-test timeouts because it helps
make our testing results more reliable and consistent.
My previous concern is that it might not be a good idea to intentionally
let the test hang in AZP
Just to add to Dong Lin's list of cons of allowing timeout:
- Any timeout value that you manually set is arbitrary. If it's set too
low, you get test instabilities. What too low means depends on numerous
factors, such as hardware and current utilization (especially I/O). If you
run in VMs and the V
I think we do capture the INFO logs of the test runs on AZP.
I am also not sure whether we really caught slow tests with Junit's timeout
rule before. I think the default is usually to increase the timeout to make
the test pass. One way to find slow tests is to measure the time and look
at the outl
There is one more point that may be useful to consider here.
In order to debug deadlock that is not easily reproducible, it is likely
not sufficient to see only the thread dump to figure out the root cause. We
likely need to enable the INFO level logging. Since AZP does not provide
INFO level logg
Just to make sure I understand the proposal correctly: is the proposal to
disallow the usage of @Test(timeout=...) for Flink Junit tests?
Here is my understanding of the pros/cons according to the discussion so
far.
Pros of allowing timeout:
1) When there are tests that are unreasonably slow, it
Assuming that not many tests deadlock I think it should be fine to simply
let the build process deadlock. Even if multiple tests fail consistently,
then one would see them one after another. That way we wouldn't have to
build some extra tooling. Moreover, the behaviour would be consistent on
the lo
I was actually recently wondering if we shouldn't rather use timeouts more
aggressively in JUnit.
There was recently a case where a number of tests accidentally ran for 5
minutes, because a timeout was increased to 5 minutes.
If we had a global limit of 1 minute per test, we would have caught this
+1. I think this rule makes a lot of sense.
Cheers,
Till
On Mon, Apr 26, 2021 at 10:08 AM Arvid Heise wrote:
> +1 from my side.
>
> We should probably double-check if we really need 4h timeouts on test tasks
> in AZP. It feels like 2h be enough.
>
> On Mon, Apr 26, 2021 at 9:54 AM Dawid Wysakow
+1 from my side.
We should probably double-check if we really need 4h timeouts on test tasks
in AZP. It feels like 2h be enough.
On Mon, Apr 26, 2021 at 9:54 AM Dawid Wysakowicz
wrote:
> Hi devs!
>
> I wanted to bring up something that was discussed in a few independent
> groups of people in th
Hi devs!
I wanted to bring up something that was discussed in a few independent
groups of people in the past days. I'd like to revise using timeouts in
our JUnit tests. The suggestion would be not to use them anymore. The
problem with timeouts is that we have no thread dump and stack traces of
the
16 matches
Mail list logo
