The class contains single test method, which runs single job (the job is quite complex), then waits for job being running after that waits for data being populated in output topic, and this doesn't happen during 5 minutes (test timeout). Tried under debugger, set breakpoint in Kafka record deserializer it is hit but very slow, roughly 3 records per 5 minute (the topic was pre-populated)
No table/sql API, only stream API ________________________________ From: Chesnay Schepler <ches...@apache.org> Sent: Wednesday, September 7, 2022 5:20 AM To: Alexey Trenikhun <yen...@msn.com>; David Jost <david.j...@uniberg.com>; Matthias Pohl <matthias.p...@aiven.io> Cc: user@flink.apache.org <user@flink.apache.org> Subject: Re: Slow Tests in Flink 1.15 The test that gotten slow; how many test cases does it actually contain / how many jobs does it actually run? Are these tests using the table/sql API? On 07/09/2022 14:15, Alexey Trenikhun wrote: We are also observing extreme slow down (5+ minutes vs 15 seconds) in 1 of 2 integration tests . Both tests use Kafka. The slow test uses org.apache.flink.runtime.minicluster.TestingMiniCluster, this test tests complete job, which consumes and produces Kafka messages. Not affected test extends org.apache.flink.test.util.AbstractTestBase which uses MiniClusterWithClientResource, this test is simpler and only produce Kafka messages. Thanks, Alexey ________________________________ From: Matthias Pohl via user <user@flink.apache.org><mailto:user@flink.apache.org> Sent: Tuesday, September 6, 2022 6:36 AM To: David Jost <david.j...@uniberg.com><mailto:david.j...@uniberg.com> Cc: user@flink.apache.org<mailto:user@flink.apache.org> <user@flink.apache.org><mailto:user@flink.apache.org> Subject: Re: Slow Tests in Flink 1.15 Hi David, I guess, you're referring to [1]. But as Chesnay already pointed out in the previous thread: It would be helpful to get more insights into what exactly your tests are executing (logs, code, ...). That would help identifying the cause. > Can you give us a more complete stacktrace so we can see what call in > Flink is waiting for something? > > Does this happen to all of your tests? > Can you provide us with an example that we can try ourselves? If not, > can you describe the test structure (e.g., is it using a > MiniClusterResource). Matthias [1] https://lists.apache.org/thread/yhhprwyf29kgypzzqdmjgft4qs25yyhk On Mon, Sep 5, 2022 at 4:59 PM David Jost <david.j...@uniberg.com<mailto:david.j...@uniberg.com>> wrote: Hi, we were going to upgrade our application from Flink 1.14.4 to Flink 1.15.2, when we noticed, that all our job tests, using a MiniClusterWithClientResource, are multiple times slower in 1.15 than before in 1.14. I, unfortunately, have not found mentions in that regard in the changelog or documentation. The slowdown is rather extreme I hope to find a solution to this. I saw it mentioned once in the mailing list, but there was no (public) outcome to it. I would appreciate any help on this. Thank you in advance. Best David
