[
https://issues.apache.org/jira/browse/FLINK-19951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237090#comment-17237090
]
Shuiqiang Chen commented on FLINK-19951:
----------------------------------------
Hi everyone, as illustrated in the comments above, I have added a timeout
limitation for reading output result from Kafka so that it won't stuck forever.
And it will print logs for JM and TM for debug purpose if the read data is not
as expected. Please refer to [this
pr|https://github.com/apache/flink/pull/14068]
Overall, to figure out the root cause, we need to observe for a little while
with more debug information for test failures.
> PyFlink end-to-end test stuck in "Reading kafka messages"
> ---------------------------------------------------------
>
> Key: FLINK-19951
> URL: https://issues.apache.org/jira/browse/FLINK-19951
> Project: Flink
> Issue Type: Bug
> Components: API / Python
> Affects Versions: 1.12.0
> Reporter: Robert Metzger
> Assignee: Shuiqiang Chen
> Priority: Major
> Labels: test-stability
> Fix For: 1.12.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=8837&view=logs&j=c88eea3b-64a0-564d-0031-9fdcd7b8abee&t=ff888d9b-cd34-53cc-d90f-3e446d355529
> {code}
> 2020-11-03T08:18:10.2935249Z Nov 03 08:18:10 Test PyFlink DataStream job:
> 2020-11-03T08:18:10.2936216Z Nov 03 08:18:10 Preparing Kafka...
> 2020-11-03T08:18:10.2948091Z Nov 03 08:18:10 Downloading Kafka from
> https://archive.apache.org/dist/kafka/2.2.0/kafka_2.12-2.2.0.tgz
> 2020-11-03T08:18:10.3024006Z % Total % Received % Xferd Average Speed
> Time Time Time Current
> 2020-11-03T08:18:10.3024610Z Dload Upload
> Total Spent Left Speed
> 2020-11-03T08:18:10.3024891Z
> 2020-11-03T08:18:10.6563956Z 0 0 0 0 0 0 0 0
> --:--:-- --:--:-- --:--:-- 0
> 2020-11-03T08:18:11.6568328Z 0 54.3M 0 32768 0 0 92275 0
> 0:10:18 --:--:-- 0:10:18 92044
> 2020-11-03T08:18:12.6540430Z 11 54.3M 11 6272k 0 0 4626k 0
> 0:00:12 0:00:01 0:00:11 4625k
> 2020-11-03T08:18:13.6585146Z 23 54.3M 23 12.6M 0 0 5521k 0
> 0:00:10 0:00:02 0:00:08 5521k
> 2020-11-03T08:18:14.6558377Z 36 54.3M 36 19.7M 0 0 6018k 0
> 0:00:09 0:00:03 0:00:06 6017k
> 2020-11-03T08:18:15.6593118Z 49 54.3M 49 26.7M 0 0 6297k 0
> 0:00:08 0:00:04 0:00:04 6297k
> 2020-11-03T08:18:16.6534444Z 62 54.3M 62 34.0M 0 0 6515k 0
> 0:00:08 0:00:05 0:00:03 6973k
> 2020-11-03T08:18:17.6544951Z 76 54.3M 76 41.8M 0 0 6747k 0
> 0:00:08 0:00:06 0:00:02 7322k
> 2020-11-03T08:18:18.2448109Z 91 54.3M 91 49.7M 0 0 6923k 0
> 0:00:08 0:00:07 0:00:01 7584k
> 2020-11-03T08:18:18.2450531Z 100 54.3M 100 54.3M 0 0 7010k 0
> 0:00:07 0:00:07 --:--:-- 7737k
> 2020-11-03T08:18:20.2751451Z Nov 03 08:18:20 Zookeeper Server has been
> started ...
> 2020-11-03T08:18:22.0064118Z Nov 03 08:18:22 Waiting for broker...
> 2020-11-03T08:18:25.4758082Z Nov 03 08:18:25 Created topic
> test-python-data-stream-source.
> 2020-11-03T08:18:25.8324767Z Nov 03 08:18:25 Sending messages to Kafka...
> 2020-11-03T08:18:35.2954788Z Nov 03 08:18:35 >>>>>>>>>>Created topic
> test-python-data-stream-sink.
> 2020-11-03T08:18:54.8314099Z Nov 03 08:18:54 Job has been submitted with
> JobID 1b0c317b47c69ee600937e1715ad9cce
> 2020-11-03T08:18:54.8348757Z Nov 03 08:18:54 Reading kafka messages...
> 2020-11-03T08:53:10.5246998Z
> ==========================================================================================
> 2020-11-03T08:53:10.5249381Z === WARNING: This E2E Run took already 80% of
> the allocated time budget of 250 minutes ===
> 2020-11-03T08:53:10.5251343Z
> ==========================================================================================
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
