[
https://issues.apache.org/jira/browse/FLINK-24764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17440257#comment-17440257
]
Till Rohrmann commented on FLINK-24764:
---------------------------------------
Another instance:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=26093&view=logs&j=821b528f-1eed-5598-a3b4-7f748b13f261&t=6bb545dd-772d-5d8c-f258-f5085fba3295&l=22613
> pyflink/table/tests/test_udf.py hang on Azure
> ---------------------------------------------
>
> Key: FLINK-24764
> URL: https://issues.apache.org/jira/browse/FLINK-24764
> Project: Flink
> Issue Type: Bug
> Components: API / Python
> Affects Versions: 1.12.5
> Reporter: Till Rohrmann
> Priority: Critical
> Labels: test-stability
> Fix For: 1.15.0, 1.14.1, 1.13.4
>
>
> {{pyflink/table/tests/test_udf.py}} seems to hang on Azure.
> {code}
> 2021-11-04T03:12:01.4537829Z py36-cython run-test: commands[3] | pytest
> --durations=20
> 2021-11-04T03:12:03.6955063Z ============================= test session
> starts ==============================
> 2021-11-04T03:12:03.6957461Z platform linux -- Python 3.6.13, pytest-6.2.5,
> py-1.10.0, pluggy-1.0.0
> 2021-11-04T03:12:03.6959285Z cachedir: .tox/py36-cython/.pytest_cache
> 2021-11-04T03:12:03.6960653Z rootdir: /__w/1/s/flink-python
> 2021-11-04T03:12:03.6961356Z collected 690 items
> 2021-11-04T03:12:03.6961755Z
> 2021-11-04T03:12:04.6615796Z pyflink/common/tests/test_configuration.py
> .......... [ 1%]
> 2021-11-04T03:12:04.9315499Z pyflink/common/tests/test_execution_config.py
> ....................... [ 4%]
> 2021-11-04T03:12:05.4226061Z
> pyflink/common/tests/test_serialization_schemas.py ... [
> 5%]
> 2021-11-04T03:12:05.8920762Z pyflink/common/tests/test_typeinfo.py ...
> [ 5%]
> 2021-11-04T03:12:10.3843622Z
> pyflink/dataset/tests/test_execution_environment.py ...s..... [
> 6%]
> 2021-11-04T03:12:10.4385641Z
> pyflink/dataset/tests/test_execution_environment_completeness.py . [
> 7%]
> 2021-11-04T03:12:10.5390180Z
> pyflink/datastream/tests/test_check_point_config.py ........... [
> 8%]
> 2021-11-04T03:12:20.1148835Z pyflink/datastream/tests/test_connectors.py ...
> [ 9%]
> 2021-11-04T03:13:12.4436977Z pyflink/datastream/tests/test_data_stream.py
> ........................... [ 13%]
> 2021-11-04T03:13:22.6815256Z ............
> [ 14%]
> 2021-11-04T03:13:22.9777981Z pyflink/datastream/tests/test_state_backend.py
> .............. [ 16%]
> 2021-11-04T03:13:33.4281095Z
> pyflink/datastream/tests/test_stream_execution_environment.py .......... [
> 18%]
> 2021-11-04T03:13:45.3707210Z .....s.................
> [ 21%]
> 2021-11-04T03:13:45.5100419Z
> pyflink/datastream/tests/test_stream_execution_environment_completeness.py .
> [ 21%]
> 2021-11-04T03:13:45.5107357Z
> [ 21%]
> 2021-11-04T03:13:45.5824541Z pyflink/fn_execution/tests/test_coders.py
> sssssssssssssssssssss [ 24%]
> 2021-11-04T03:13:45.6311670Z pyflink/fn_execution/tests/test_fast_coders.py
> ................... [ 27%]
> 2021-11-04T03:13:45.6480686Z
> pyflink/fn_execution/tests/test_flink_fn_execution_pb2_synced.py . [
> 27%]
> 2021-11-04T03:13:48.3033527Z
> pyflink/fn_execution/tests/test_process_mode_boot.py ... [
> 28%]
> 2021-11-04T03:13:48.3169538Z pyflink/metrics/tests/test_metric.py .....
> [ 28%]
> 2021-11-04T03:13:48.3928810Z pyflink/ml/tests/test_ml_environment.py ...
> [ 29%]
> 2021-11-04T03:13:48.4381082Z pyflink/ml/tests/test_ml_environment_factory.py
> ... [ 29%]
> 2021-11-04T03:13:48.4696143Z pyflink/ml/tests/test_params.py .............
> [ 31%]
> 2021-11-04T03:13:48.5140301Z pyflink/ml/tests/test_pipeline.py ....
> [ 32%]
> 2021-11-04T03:13:50.2573824Z pyflink/ml/tests/test_pipeline_it_case.py ...
> [ 32%]
> 2021-11-04T03:13:50.3598135Z pyflink/ml/tests/test_pipeline_stage.py ..
> [ 32%]
> 2021-11-04T03:14:18.5397420Z pyflink/table/tests/test_aggregate.py .........
> [ 34%]
> 2021-11-04T03:14:20.1852937Z pyflink/table/tests/test_calc.py .......
> [ 35%]
> 2021-11-04T03:14:21.3674525Z pyflink/table/tests/test_catalog.py
> .................................... [ 40%]
> 2021-11-04T03:14:22.4375814Z ...........................................
> [ 46%]
> 2021-11-04T03:14:22.4966492Z pyflink/table/tests/test_catalog_completeness.py
> ........ [ 47%]
> 2021-11-04T03:14:22.7401730Z pyflink/table/tests/test_column_operation.py
> .... [ 48%]
> 2021-11-04T03:14:23.0096076Z pyflink/table/tests/test_correlate.py ....
> [ 48%]
> 2021-11-04T03:14:45.9507861Z pyflink/table/tests/test_dependency.py ........
> [ 50%]
> 2021-11-04T03:14:46.2722163Z pyflink/table/tests/test_descriptor.py
> ................................. [ 54%]
> 2021-11-04T03:14:47.8354647Z
> ......................................................... [
> 63%]
> 2021-11-04T03:14:47.8736949Z pyflink/table/tests/test_distinct.py .
> [ 63%]
> 2021-11-04T03:14:47.8937508Z
> pyflink/table/tests/test_environment_completeness.py . [
> 63%]
> 2021-11-04T03:14:47.9173952Z pyflink/table/tests/test_environment_settings.py
> .... [ 64%]
> 2021-11-04T03:14:47.9299408Z
> pyflink/table/tests/test_environment_settings_completeness.py .. [
> 64%]
> 2021-11-04T03:14:48.1290014Z pyflink/table/tests/test_explain.py .
> [ 64%]
> 2021-11-04T03:14:48.2802920Z pyflink/table/tests/test_expression.py ..
> [ 64%]
> 2021-11-04T03:14:48.3411951Z
> pyflink/table/tests/test_expression_completeness.py . [
> 64%]
> 2021-11-04T03:14:48.3659747Z
> pyflink/table/tests/test_expressions_completeness.py . [
> 65%]
> 2021-11-04T03:14:48.7382764Z pyflink/table/tests/test_join.py ......
> [ 65%]
> 2021-11-04T03:15:07.8971840Z pyflink/table/tests/test_pandas_conversion.py
> ................ [ 68%]
> 2021-11-04T03:15:49.5061569Z pyflink/table/tests/test_pandas_udaf.py
> .............. [ 70%]
> 2021-11-04T03:16:35.0412479Z pyflink/table/tests/test_pandas_udf.py
> ............. [ 72%]
> 2021-11-04T03:16:35.2174275Z pyflink/table/tests/test_schema_operation.py ..
> [ 72%]
> 2021-11-04T03:16:35.5570472Z pyflink/table/tests/test_set_operation.py ......
> [ 73%]
> 2021-11-04T03:16:35.9440722Z pyflink/table/tests/test_shell_example.py ..
> [ 73%]
> 2021-11-04T03:16:36.0818236Z pyflink/table/tests/test_sort.py ...
> [ 74%]
> 2021-11-04T03:16:42.3128606Z pyflink/table/tests/test_sql.py ......
> [ 74%]
> 2021-11-04T03:16:42.3393973Z pyflink/table/tests/test_table_completeness.py .
> [ 75%]
> 2021-11-04T03:16:42.4322263Z pyflink/table/tests/test_table_config.py
> ......... [ 76%]
> 2021-11-04T03:16:42.4497849Z
> pyflink/table/tests/test_table_config_completeness.py . [
> 76%]
> 2021-11-04T03:16:48.7891322Z
> pyflink/table/tests/test_table_environment_api.py ...................... [
> 79%]
> 2021-11-04T03:16:58.1699567Z .........................................
> [ 85%]
> 2021-11-04T03:16:58.8107194Z pyflink/table/tests/test_table_schema.py
> ............ [ 87%]
> 2021-11-04T03:17:01.4033597Z pyflink/table/tests/test_types.py
> ................................... [ 92%]
> 2021-11-04T03:33:04.0138220Z pyflink/table/tests/test_udf.py
> ............................==============================================================================
> 2021-11-04T03:33:04.0138913Z Process produced no output for 900 seconds.
> 2021-11-04T03:33:04.0139623Z
> ==============================================================================
> 2021-11-04T03:33:04.0145949Z
> ==============================================================================
> 2021-11-04T03:33:04.0146967Z The following Java processes are running (JPS)
> 2021-11-04T03:33:04.0147420Z
> ==============================================================================
> 2021-11-04T03:33:04.0218249Z Picked up JAVA_TOOL_OPTIONS:
> -XX:+HeapDumpOnOutOfMemoryError
> 2021-11-04T03:33:04.2000801Z 77383 PythonGatewayServer
> 2021-11-04T03:33:04.2001279Z 98730 Jps
> 2021-11-04T03:33:04.2046186Z
> ==============================================================================
> 2021-11-04T03:33:04.2046623Z Printing stack trace of Java process 77383
> 2021-11-04T03:33:04.2050853Z
> ==============================================================================
> 2021-11-04T03:33:04.2092916Z Picked up JAVA_TOOL_OPTIONS:
> -XX:+HeapDumpOnOutOfMemoryError
> 2021-11-04T03:33:04.5640129Z 2021-11-04 03:33:04
> 2021-11-04T03:33:04.5641239Z Full thread dump OpenJDK 64-Bit Server VM
> (25.282-b08 mixed mode):
> 2021-11-04T03:33:04.5641705Z
> 2021-11-04T03:33:04.5645703Z "Attach Listener" #14449 daemon prio=9 os_prio=0
> tid=0x00007f742026c800 nid=0x181c5 runnable [0x0000000000000000]
> 2021-11-04T03:33:04.5646878Z java.lang.Thread.State: RUNNABLE
> 2021-11-04T03:33:04.5647097Z
> 2021-11-04T03:33:04.5649168Z "flink-akka.actor.default-dispatcher-94" #14448
> prio=5 os_prio=0 tid=0x00007f740c04f800 nid=0x181a0 waiting on condition
> [0x00007f73fbbfc000]
> 2021-11-04T03:33:04.5649828Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.5653249Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.5654812Z - parking to wait for <0x00000000e2e88048> (a
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinPool)
> 2021-11-04T03:33:04.5656448Z at
> akka.dispatch.forkjoin.ForkJoinPool.scan(ForkJoinPool.java:2075)
> 2021-11-04T03:33:04.5659941Z at
> akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> 2021-11-04T03:33:04.5662500Z at
> akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> 2021-11-04T03:33:04.5662966Z
> 2021-11-04T03:33:04.5674146Z "flink-taskexecutor-io-thread-1" #14368 daemon
> prio=5 os_prio=0 tid=0x00007f740c02d000 nid=0x17fc0 waiting on condition
> [0x00007f73d7ffe000]
> 2021-11-04T03:33:04.5678969Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.5679628Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.5681892Z - parking to wait for <0x00000000e35b5128> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.5684484Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.5687799Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> 2021-11-04T03:33:04.5688709Z at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> 2021-11-04T03:33:04.5689650Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.5690229Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.5694053Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.5694726Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.5694904Z
> 2021-11-04T03:33:04.5695729Z "Flink-DispatcherRestEndpoint-thread-4" #14358
> daemon prio=5 os_prio=0 tid=0x00007f741c34d000 nid=0x17f8e waiting on
> condition [0x00007f73d66e7000]
> 2021-11-04T03:33:04.5697814Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.5700918Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.5770674Z - parking to wait for <0x00000000e362a890> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.5771503Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.5772202Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> 2021-11-04T03:33:04.5772995Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
> 2021-11-04T03:33:04.5773730Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> 2021-11-04T03:33:04.5776036Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.5776924Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.5777585Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.5799645Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.5800887Z
> 2021-11-04T03:33:04.5809590Z "Flink-DispatcherRestEndpoint-thread-3" #14357
> daemon prio=5 os_prio=0 tid=0x00007f73ebb62800 nid=0x17f8d waiting on
> condition [0x00007f73f9c24000]
> 2021-11-04T03:33:04.5813500Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.5814024Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.5827398Z - parking to wait for <0x00000000e362a890> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.5828319Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.5829012Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> 2021-11-04T03:33:04.5834268Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
> 2021-11-04T03:33:04.5836773Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> 2021-11-04T03:33:04.5839612Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.5852571Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.5863778Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.5870477Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.5878743Z
> 2021-11-04T03:33:04.5884225Z "Flink-DispatcherRestEndpoint-thread-2" #14355
> daemon prio=5 os_prio=0 tid=0x00007f73d99cd000 nid=0x17f81 waiting on
> condition [0x00007f73fb988000]
> 2021-11-04T03:33:04.5884923Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.5885509Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.5886775Z - parking to wait for <0x00000000e362a890> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.5887954Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.5888767Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> 2021-11-04T03:33:04.5889631Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
> 2021-11-04T03:33:04.5890311Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> 2021-11-04T03:33:04.5891062Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.5891600Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.5892147Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.5892998Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.5893210Z
> 2021-11-04T03:33:04.5894054Z "grpc-nio-worker-ELG-201-2" #14350 daemon prio=5
> os_prio=0 tid=0x000055ffb5fc0800 nid=0x17f57 runnable [0x00007f73e5ff1000]
> 2021-11-04T03:33:04.5894625Z java.lang.Thread.State: RUNNABLE
> 2021-11-04T03:33:04.5895060Z at
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> 2021-11-04T03:33:04.5896077Z at
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> 2021-11-04T03:33:04.5896919Z at
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> 2021-11-04T03:33:04.5897386Z at
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> 2021-11-04T03:33:04.5898227Z - locked <0x00000000e467bfd8> (a
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.channel.nio.SelectedSelectionKeySet)
> 2021-11-04T03:33:04.5898950Z - locked <0x00000000e467bd80> (a
> java.util.Collections$UnmodifiableSet)
> 2021-11-04T03:33:04.5899586Z - locked <0x00000000e467bc68> (a
> sun.nio.ch.EPollSelectorImpl)
> 2021-11-04T03:33:04.5900034Z at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> 2021-11-04T03:33:04.5900822Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
> 2021-11-04T03:33:04.5901827Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:824)
> 2021-11-04T03:33:04.5902621Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
> 2021-11-04T03:33:04.5903292Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
> 2021-11-04T03:33:04.5903977Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> 2021-11-04T03:33:04.5904651Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> 2021-11-04T03:33:04.5905194Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.5905395Z
> 2021-11-04T03:33:04.5906057Z "grpc-nio-worker-ELG-201-1" #14348 daemon prio=5
> os_prio=0 tid=0x000055ffb5fc5000 nid=0x17f51 runnable [0x00007f73d57d8000]
> 2021-11-04T03:33:04.5906610Z java.lang.Thread.State: RUNNABLE
> 2021-11-04T03:33:04.5907178Z at
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> 2021-11-04T03:33:04.5907929Z at
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> 2021-11-04T03:33:04.5908433Z at
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> 2021-11-04T03:33:04.5908940Z at
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> 2021-11-04T03:33:04.5909765Z - locked <0x00000000e4678540> (a
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.channel.nio.SelectedSelectionKeySet)
> 2021-11-04T03:33:04.5910654Z - locked <0x00000000e46782e8> (a
> java.util.Collections$UnmodifiableSet)
> 2021-11-04T03:33:04.5911421Z - locked <0x00000000e46781d0> (a
> sun.nio.ch.EPollSelectorImpl)
> 2021-11-04T03:33:04.5912003Z at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> 2021-11-04T03:33:04.5912577Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
> 2021-11-04T03:33:04.5913370Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:824)
> 2021-11-04T03:33:04.5913958Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
> 2021-11-04T03:33:04.5914591Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
> 2021-11-04T03:33:04.5915243Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> 2021-11-04T03:33:04.5915873Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> 2021-11-04T03:33:04.5916383Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.5916550Z
> 2021-11-04T03:33:04.5917238Z "TableResult-await-thread" #14346 daemon prio=5
> os_prio=0 tid=0x00007f73db096800 nid=0x17f39 waiting on condition
> [0x00007f73ec1fb000]
> 2021-11-04T03:33:04.5917741Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.5918086Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.5918688Z - parking to wait for <0x00000000e47415a0> (a
> java.util.concurrent.CompletableFuture$Signaller)
> 2021-11-04T03:33:04.5919183Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.5919680Z at
> java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1707)
> 2021-11-04T03:33:04.5920180Z at
> java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
> 2021-11-04T03:33:04.5920675Z at
> java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1742)
> 2021-11-04T03:33:04.5921166Z at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
> 2021-11-04T03:33:04.5921686Z at
> org.apache.flink.table.api.internal.InsertResultIterator.hasNext(InsertResultIterator.java:54)
> 2021-11-04T03:33:04.5922299Z at
> org.apache.flink.table.api.internal.TableResultImpl$CloseableRowIteratorWrapper.hasNext(TableResultImpl.java:350)
> 2021-11-04T03:33:04.5922946Z at
> org.apache.flink.table.api.internal.TableResultImpl$CloseableRowIteratorWrapper.isFirstRowReady(TableResultImpl.java:363)
> 2021-11-04T03:33:04.5923556Z at
> org.apache.flink.table.api.internal.TableResultImpl.lambda$awaitInternal$1(TableResultImpl.java:110)
> 2021-11-04T03:33:04.5924096Z at
> org.apache.flink.table.api.internal.TableResultImpl$$Lambda$1757/1188663962.run(Unknown
> Source)
> 2021-11-04T03:33:04.5924597Z at
> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
> 2021-11-04T03:33:04.5925108Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> 2021-11-04T03:33:04.5925613Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.5926042Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.5926206Z
> 2021-11-04T03:33:04.5926848Z "grpc-nio-boss-ELG-200-1" #14345 daemon prio=5
> os_prio=0 tid=0x00007f742c07e000 nid=0x17f38 runnable [0x00007f73fbcfd000]
> 2021-11-04T03:33:04.5927463Z java.lang.Thread.State: RUNNABLE
> 2021-11-04T03:33:04.5927834Z at
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> 2021-11-04T03:33:04.5928411Z at
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> 2021-11-04T03:33:04.5958791Z at
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> 2021-11-04T03:33:04.5959911Z at
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> 2021-11-04T03:33:04.5960888Z - locked <0x00000000e46747e8> (a
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.channel.nio.SelectedSelectionKeySet)
> 2021-11-04T03:33:04.5961655Z - locked <0x00000000e4664580> (a
> java.util.Collections$UnmodifiableSet)
> 2021-11-04T03:33:04.5962470Z - locked <0x00000000e4664468> (a
> sun.nio.ch.EPollSelectorImpl)
> 2021-11-04T03:33:04.5963076Z at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> 2021-11-04T03:33:04.5963918Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
> 2021-11-04T03:33:04.5968276Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:824)
> 2021-11-04T03:33:04.5968991Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
> 2021-11-04T03:33:04.5969621Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
> 2021-11-04T03:33:04.5970277Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> 2021-11-04T03:33:04.5970914Z at
> org.apache.beam.vendor.grpc.v1p26p0.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> 2021-11-04T03:33:04.5971429Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.5971613Z
> 2021-11-04T03:33:04.5973350Z "Source: PythonInputFormatTableSource(a, b, c,
> d, e, f, g, h, i, j, k, l, m, n, o, p, q) ->
> SourceConversion(table=[default_catalog.default_database.Unregistered_TableSource_856415295,
> source: [PythonInputFormatTableSource(a, b, c, d, e, f, g, h, i, j, k, l, m,
> n, o, p, q)]], fields=[a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q]) ->
> BatchExecPythonCalc -> SinkConversionToRow -> Map -> Sink: Unnamed (2/2)#0"
> #14340 prio=5 os_prio=0 tid=0x00007f7404117000 nid=0x17f37 waiting on
> condition [0x00007f73fbafa000]
> 2021-11-04T03:33:04.5974714Z java.lang.Thread.State: TIMED_WAITING
> (parking)
> 2021-11-04T03:33:04.5975082Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.5975713Z - parking to wait for <0x00000000e625df20> (a
> java.util.concurrent.CompletableFuture$Signaller)
> 2021-11-04T03:33:04.5976287Z at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> 2021-11-04T03:33:04.5976795Z at
> java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1709)
> 2021-11-04T03:33:04.5977307Z at
> java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
> 2021-11-04T03:33:04.5977797Z at
> java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1788)
> 2021-11-04T03:33:04.5978285Z at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
> 2021-11-04T03:33:04.5978832Z at
> org.apache.beam.runners.fnexecution.control.MapControlClientPool.getClient(MapControlClientPool.java:69)
> 2021-11-04T03:33:04.5979400Z at
> org.apache.beam.runners.fnexecution.control.MapControlClientPool$$Lambda$1346/2006768656.take(Unknown
> Source)
> 2021-11-04T03:33:04.5980008Z at
> org.apache.beam.runners.fnexecution.environment.ProcessEnvironmentFactory.createEnvironment(ProcessEnvironmentFactory.java:113)
> 2021-11-04T03:33:04.5980700Z at
> org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:248)
> 2021-11-04T03:33:04.5990463Z at
> org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:227)
> 2021-11-04T03:33:04.5993732Z at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
> 2021-11-04T03:33:04.5994709Z at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
> 2021-11-04T03:33:04.5995637Z at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
> 2021-11-04T03:33:04.5999577Z - locked <0x00000000e45f7d78> (a
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$StrongEntry)
> 2021-11-04T03:33:04.6000247Z at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
> 2021-11-04T03:33:04.6004552Z at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.get(LocalCache.java:3952)
> 2021-11-04T03:33:04.6012324Z at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
> 2021-11-04T03:33:04.6013339Z at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958)
> 2021-11-04T03:33:04.6015848Z at
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4964)
> 2021-11-04T03:33:04.6016673Z at
> org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:447)
> 2021-11-04T03:33:04.6022568Z at
> org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory.<init>(DefaultJobBundleFactory.java:432)
> 2021-11-04T03:33:04.6030814Z at
> org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.forStage(DefaultJobBundleFactory.java:299)
> 2021-11-04T03:33:04.6034265Z at
> org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.createStageBundleFactory(BeamPythonFunctionRunner.java:413)
> 2021-11-04T03:33:04.6048364Z at
> org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.open(BeamPythonFunctionRunner.java:259)
> 2021-11-04T03:33:04.6060827Z at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.open(AbstractPythonFunctionOperator.java:113)
> 2021-11-04T03:33:04.6070904Z at
> org.apache.flink.table.runtime.operators.python.AbstractStatelessFunctionOperator.open(AbstractStatelessFunctionOperator.java:116)
> 2021-11-04T03:33:04.6072240Z at
> org.apache.flink.table.runtime.operators.python.scalar.AbstractPythonScalarFunctionOperator.open(AbstractPythonScalarFunctionOperator.java:88)
> 2021-11-04T03:33:04.6073442Z at
> org.apache.flink.table.runtime.operators.python.scalar.AbstractRowDataPythonScalarFunctionOperator.open(AbstractRowDataPythonScalarFunctionOperator.java:70)
> 2021-11-04T03:33:04.6074282Z at
> org.apache.flink.table.runtime.operators.python.scalar.RowDataPythonScalarFunctionOperator.open(RowDataPythonScalarFunctionOperator.java:59)
> 2021-11-04T03:33:04.6083149Z at
> org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:433)
> 2021-11-04T03:33:04.6088550Z at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$2(StreamTask.java:548)
> 2021-11-04T03:33:04.6090437Z at
> org.apache.flink.streaming.runtime.tasks.StreamTask$$Lambda$1226/2012103952.run(Unknown
> Source)
> 2021-11-04T03:33:04.6091341Z at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:93)
> 2021-11-04T03:33:04.6092251Z - locked <0x00000000e4222bb8> (a
> java.lang.Object)
> 2021-11-04T03:33:04.6095727Z at
> org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:538)
> 2021-11-04T03:33:04.6101449Z at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:578)
> 2021-11-04T03:33:04.6102054Z at
> org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:758)
> 2021-11-04T03:33:04.6102521Z at
> org.apache.flink.runtime.taskmanager.Task.run(Task.java:573)
> 2021-11-04T03:33:04.6102957Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6103128Z
> 2021-11-04T03:33:04.6103539Z "CloseableReaperThread" #14339 daemon prio=5
> os_prio=0 tid=0x00007f742c078800 nid=0x17f36 in Object.wait()
> [0x00007f73e52e6000]
> 2021-11-04T03:33:04.6104284Z java.lang.Thread.State: WAITING (on object
> monitor)
> 2021-11-04T03:33:04.6104666Z at java.lang.Object.wait(Native Method)
> 2021-11-04T03:33:04.6105065Z at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
> 2021-11-04T03:33:04.6105836Z - locked <0x00000000e41c9ad0> (a
> java.lang.ref.ReferenceQueue$Lock)
> 2021-11-04T03:33:04.6106290Z at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
> 2021-11-04T03:33:04.6106850Z at
> org.apache.flink.core.fs.SafetyNetCloseableRegistry$CloseableReaperThread.run(SafetyNetCloseableRegistry.java:208)
> 2021-11-04T03:33:04.6107182Z
> 2021-11-04T03:33:04.6107844Z "Flink-MetricRegistry-thread-1" #14337 daemon
> prio=5 os_prio=0 tid=0x00007f74042ab800 nid=0x17f34 waiting on condition
> [0x00007f73e5def000]
> 2021-11-04T03:33:04.6108539Z java.lang.Thread.State: TIMED_WAITING
> (parking)
> 2021-11-04T03:33:04.6108905Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6109616Z - parking to wait for <0x00000000e2df2050> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6110171Z at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> 2021-11-04T03:33:04.6110755Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> 2021-11-04T03:33:04.6111409Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
> 2021-11-04T03:33:04.6112040Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> 2021-11-04T03:33:04.6112795Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.6113318Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.6113871Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.6114333Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6114535Z
> 2021-11-04T03:33:04.6115811Z "pool-550-thread-1" #14334 prio=5 os_prio=0
> tid=0x00007f73d99cc000 nid=0x17f31 waiting on condition [0x00007f73fa127000]
> 2021-11-04T03:33:04.6116518Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.6116911Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6117643Z - parking to wait for <0x00000000e361f420> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6118228Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.6119017Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> 2021-11-04T03:33:04.6119999Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
> 2021-11-04T03:33:04.6120797Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> 2021-11-04T03:33:04.6121351Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.6121845Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.6122355Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.6122786Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6123151Z
> 2021-11-04T03:33:04.6123812Z "jobmanager-future-thread-2" #14331 daemon
> prio=5 os_prio=0 tid=0x00007f73d99cb800 nid=0x17f2e waiting on condition
> [0x00007f740828f000]
> 2021-11-04T03:33:04.6124528Z java.lang.Thread.State: TIMED_WAITING
> (parking)
> 2021-11-04T03:33:04.6125558Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6126316Z - parking to wait for <0x00000000e3b22c98> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6126910Z at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> 2021-11-04T03:33:04.6128139Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> 2021-11-04T03:33:04.6129174Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
> 2021-11-04T03:33:04.6129829Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> 2021-11-04T03:33:04.6130399Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.6130984Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.6131525Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.6131982Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6132267Z
> 2021-11-04T03:33:04.6133184Z "mini-cluster-io-thread-8" #14322 daemon prio=5
> os_prio=0 tid=0x00007f742026c000 nid=0x17f25 waiting on condition
> [0x00007f73f971f000]
> 2021-11-04T03:33:04.6133699Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.6134242Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6134932Z - parking to wait for <0x00000000e35a9ea0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6135678Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.6136408Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> 2021-11-04T03:33:04.6137026Z at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> 2021-11-04T03:33:04.6137906Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.6138792Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.6139905Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.6140436Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6140647Z
> 2021-11-04T03:33:04.6141438Z "mini-cluster-io-thread-7" #14321 daemon prio=5
> os_prio=0 tid=0x00007f742027c000 nid=0x17f24 waiting on condition
> [0x00007f73fa329000]
> 2021-11-04T03:33:04.6142038Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.6142461Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6143244Z - parking to wait for <0x00000000e35a9ea0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6143876Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.6144552Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> 2021-11-04T03:33:04.6145238Z at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> 2021-11-04T03:33:04.6145845Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.6146602Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.6147545Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.6148194Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6148380Z
> 2021-11-04T03:33:04.6149627Z "jobmanager-future-thread-1" #14320 daemon
> prio=5 os_prio=0 tid=0x00007f74202d6000 nid=0x17f23 waiting on condition
> [0x00007f73e60f2000]
> 2021-11-04T03:33:04.6150167Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.6150717Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6151602Z - parking to wait for <0x00000000e3b22c98> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6152361Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.6154257Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> 2021-11-04T03:33:04.6155407Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
> 2021-11-04T03:33:04.6162767Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> 2021-11-04T03:33:04.6163602Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.6164169Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.6164887Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.6165340Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6165517Z
> 2021-11-04T03:33:04.6166556Z "mini-cluster-io-thread-6" #14318 daemon prio=5
> os_prio=0 tid=0x00007f74100a5000 nid=0x17f21 waiting on condition
> [0x00007f73e56ea000]
> 2021-11-04T03:33:04.6167738Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.6168139Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6168955Z - parking to wait for <0x00000000e35a9ea0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6172169Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.6172952Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> 2021-11-04T03:33:04.6173566Z at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> 2021-11-04T03:33:04.6174453Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.6175031Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.6175615Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.6176215Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6176404Z
> 2021-11-04T03:33:04.6177549Z "mini-cluster-io-thread-5" #14317 daemon prio=5
> os_prio=0 tid=0x00007f73db096000 nid=0x17f20 waiting on condition
> [0x00007f73e68f8000]
> 2021-11-04T03:33:04.6178090Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.6178456Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6179309Z - parking to wait for <0x00000000e35a9ea0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6179844Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.6180413Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> 2021-11-04T03:33:04.6181190Z at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> 2021-11-04T03:33:04.6181704Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.6182251Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.6182986Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.6183459Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6183648Z
> 2021-11-04T03:33:04.6184521Z "mini-cluster-io-thread-4" #14316 daemon prio=5
> os_prio=0 tid=0x00007f73db095000 nid=0x17f1f waiting on condition
> [0x00007f73fad2d000]
> 2021-11-04T03:33:04.6185039Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.6185407Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6186236Z - parking to wait for <0x00000000e35a9ea0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6186950Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.6187536Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> 2021-11-04T03:33:04.6188149Z at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> 2021-11-04T03:33:04.6188661Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.6189201Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.6190228Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.6190676Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6190853Z
> 2021-11-04T03:33:04.6191556Z "mini-cluster-io-thread-3" #14315 daemon prio=5
> os_prio=0 tid=0x00007f73d99de000 nid=0x17f1e waiting on condition
> [0x00007f73e6bfb000]
> 2021-11-04T03:33:04.6192064Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.6192423Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6193287Z - parking to wait for <0x00000000e35a9ea0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6193836Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.6194572Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> 2021-11-04T03:33:04.6195172Z at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> 2021-11-04T03:33:04.6195876Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.6196588Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.6197124Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.6197580Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6197756Z
> 2021-11-04T03:33:04.6198626Z "mini-cluster-io-thread-2" #14314 daemon prio=5
> os_prio=0 tid=0x00007f73d99dd800 nid=0x17f1d waiting on condition
> [0x00007f73e6afa000]
> 2021-11-04T03:33:04.6199196Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.6199569Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6200287Z - parking to wait for <0x00000000e35a9ea0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6200837Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.6201470Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> 2021-11-04T03:33:04.6202079Z at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> 2021-11-04T03:33:04.6202597Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.6203134Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.6203674Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.6204135Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6204312Z
> 2021-11-04T03:33:04.6205185Z "Flink-DispatcherRestEndpoint-thread-1" #14312
> daemon prio=5 os_prio=0 tid=0x00007f73eb9aa800 nid=0x17f1b waiting on
> condition [0x00007f73d79f8000]
> 2021-11-04T03:33:04.6205732Z java.lang.Thread.State: TIMED_WAITING
> (parking)
> 2021-11-04T03:33:04.6206412Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6207464Z - parking to wait for <0x00000000e362a890> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6208127Z at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> 2021-11-04T03:33:04.6208819Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> 2021-11-04T03:33:04.6209589Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
> 2021-11-04T03:33:04.6210325Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> 2021-11-04T03:33:04.6211297Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.6211883Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.6213234Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.6214321Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6214505Z
> 2021-11-04T03:33:04.6215476Z "mini-cluster-io-thread-1" #14311 daemon prio=5
> os_prio=0 tid=0x00007f73eb9aa000 nid=0x17f1a waiting on condition
> [0x00007f73e41d7000]
> 2021-11-04T03:33:04.6216004Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.6216369Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6217389Z - parking to wait for <0x00000000e35a9ea0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6218284Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.6218860Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> 2021-11-04T03:33:04.6219788Z at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> 2021-11-04T03:33:04.6220321Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.6221017Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.6221543Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.6221983Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6222153Z
> 2021-11-04T03:33:04.6222862Z "flink-rest-server-netty-boss-thread-1" #14310
> daemon prio=5 os_prio=0 tid=0x00007f73f1813800 nid=0x17f19 runnable
> [0x00007f73fa82c000]
> 2021-11-04T03:33:04.6223510Z java.lang.Thread.State: RUNNABLE
> 2021-11-04T03:33:04.6223869Z at
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> 2021-11-04T03:33:04.6224269Z at
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> 2021-11-04T03:33:04.6224729Z at
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> 2021-11-04T03:33:04.6225191Z at
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> 2021-11-04T03:33:04.6225895Z - locked <0x00000000e3686b58> (a
> org.apache.flink.shaded.netty4.io.netty.channel.nio.SelectedSelectionKeySet)
> 2021-11-04T03:33:04.6226558Z - locked <0x00000000e36768f0> (a
> java.util.Collections$UnmodifiableSet)
> 2021-11-04T03:33:04.6227144Z - locked <0x00000000e36767d8> (a
> sun.nio.ch.EPollSelectorImpl)
> 2021-11-04T03:33:04.6227565Z at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> 2021-11-04T03:33:04.6227968Z at
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
> 2021-11-04T03:33:04.6228530Z at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
> 2021-11-04T03:33:04.6229160Z at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:803)
> 2021-11-04T03:33:04.6229715Z at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
> 2021-11-04T03:33:04.6230334Z at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> 2021-11-04T03:33:04.6231165Z at
> org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> 2021-11-04T03:33:04.6231662Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6231834Z
> 2021-11-04T03:33:04.6232249Z "IOManager reader thread #1" #14304 daemon
> prio=5 os_prio=0 tid=0x00007f73f1812800 nid=0x17f17 waiting on condition
> [0x00007f73d7af9000]
> 2021-11-04T03:33:04.6232741Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.6233096Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6233956Z - parking to wait for <0x00000000e35b8160> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6234654Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.6235215Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> 2021-11-04T03:33:04.6235792Z at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> 2021-11-04T03:33:04.6236678Z at
> org.apache.flink.runtime.io.disk.iomanager.IOManagerAsync$ReaderThread.run(IOManagerAsync.java:372)
> 2021-11-04T03:33:04.6236996Z
> 2021-11-04T03:33:04.6237412Z "IOManager writer thread #1" #14303 daemon
> prio=5 os_prio=0 tid=0x00007f73fd155800 nid=0x17f16 waiting on condition
> [0x00007f73e61f3000]
> 2021-11-04T03:33:04.6237919Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.6238283Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6239001Z - parking to wait for <0x00000000e35b7c38> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6239548Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.6240135Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> 2021-11-04T03:33:04.6240852Z at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> 2021-11-04T03:33:04.6241740Z at
> org.apache.flink.runtime.io.disk.iomanager.IOManagerAsync$WriterThread.run(IOManagerAsync.java:482)
> 2021-11-04T03:33:04.6242020Z
> 2021-11-04T03:33:04.6242648Z "Timer-495" #14301 daemon prio=5 os_prio=0
> tid=0x00007f73fd154800 nid=0x17f15 in Object.wait() [0x00007f73fb39c000]
> 2021-11-04T03:33:04.6243131Z java.lang.Thread.State: TIMED_WAITING (on
> object monitor)
> 2021-11-04T03:33:04.6243504Z at java.lang.Object.wait(Native Method)
> 2021-11-04T03:33:04.6243845Z at
> java.util.TimerThread.mainLoop(Timer.java:552)
> 2021-11-04T03:33:04.6244391Z - locked <0x00000000e35afe48> (a
> java.util.TaskQueue)
> 2021-11-04T03:33:04.6244779Z at java.util.TimerThread.run(Timer.java:505)
> 2021-11-04T03:33:04.6244968Z
> 2021-11-04T03:33:04.6245535Z "Timer-494" #14299 daemon prio=5 os_prio=0
> tid=0x00007f73db0fe000 nid=0x17f14 in Object.wait() [0x00007f73f961c000]
> 2021-11-04T03:33:04.6246032Z java.lang.Thread.State: TIMED_WAITING (on
> object monitor)
> 2021-11-04T03:33:04.6246398Z at java.lang.Object.wait(Native Method)
> 2021-11-04T03:33:04.6246765Z at
> java.util.TimerThread.mainLoop(Timer.java:552)
> 2021-11-04T03:33:04.6251537Z - locked <0x00000000e35adf78> (a
> java.util.TaskQueue)
> 2021-11-04T03:33:04.6252002Z at java.util.TimerThread.run(Timer.java:505)
> 2021-11-04T03:33:04.6252178Z
> 2021-11-04T03:33:04.6252585Z "BLOB Server listener at 43215" #14295 daemon
> prio=5 os_prio=0 tid=0x00007f73db0fd000 nid=0x17f13 runnable
> [0x00007f73e5beb000]
> 2021-11-04T03:33:04.6253035Z java.lang.Thread.State: RUNNABLE
> 2021-11-04T03:33:04.6253393Z at java.net.PlainSocketImpl.socketAccept(Native
> Method)
> 2021-11-04T03:33:04.6253990Z at
> java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
> 2021-11-04T03:33:04.6255227Z at
> java.net.ServerSocket.implAccept(ServerSocket.java:560)
> 2021-11-04T03:33:04.6255768Z at
> java.net.ServerSocket.accept(ServerSocket.java:528)
> 2021-11-04T03:33:04.6256883Z at
> org.apache.flink.runtime.blob.BlobServer.run(BlobServer.java:266)
> 2021-11-04T03:33:04.6257161Z
> 2021-11-04T03:33:04.6258038Z "Timer-493" #14296 daemon prio=5 os_prio=0
> tid=0x00007f73f16a4800 nid=0x17f12 in Object.wait() [0x00007f73e4de1000]
> 2021-11-04T03:33:04.6258922Z java.lang.Thread.State: TIMED_WAITING (on
> object monitor)
> 2021-11-04T03:33:04.6259379Z at java.lang.Object.wait(Native Method)
> 2021-11-04T03:33:04.6259822Z at
> java.util.TimerThread.mainLoop(Timer.java:552)
> 2021-11-04T03:33:04.6260667Z - locked <0x00000000e35ab7e0> (a
> java.util.TaskQueue)
> 2021-11-04T03:33:04.6261139Z at java.util.TimerThread.run(Timer.java:505)
> 2021-11-04T03:33:04.6261700Z
> 2021-11-04T03:33:04.6262798Z "flink-metrics-scheduler-1" #14290 prio=5
> os_prio=0 tid=0x00007f73fd2f7000 nid=0x17f0d waiting on condition
> [0x00007f73fb29b000]
> 2021-11-04T03:33:04.6263320Z java.lang.Thread.State: TIMED_WAITING
> (sleeping)
> 2021-11-04T03:33:04.6263708Z at java.lang.Thread.sleep(Native Method)
> 2021-11-04T03:33:04.6264154Z at
> akka.actor.LightArrayRevolverScheduler.waitNanos(LightArrayRevolverScheduler.scala:85)
> 2021-11-04T03:33:04.6264867Z at
> akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:265)
> 2021-11-04T03:33:04.6265701Z at
> akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235)
> 2021-11-04T03:33:04.6266655Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6266832Z
> 2021-11-04T03:33:04.6267932Z "flink-scheduler-1" #14287 prio=5 os_prio=0
> tid=0x00007f73d99d1000 nid=0x17f0a waiting on condition [0x00007f73e69f9000]
> 2021-11-04T03:33:04.6268507Z java.lang.Thread.State: TIMED_WAITING
> (sleeping)
> 2021-11-04T03:33:04.6269259Z at java.lang.Thread.sleep(Native Method)
> 2021-11-04T03:33:04.6269752Z at
> akka.actor.LightArrayRevolverScheduler.waitNanos(LightArrayRevolverScheduler.scala:85)
> 2021-11-04T03:33:04.6270573Z at
> akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:265)
> 2021-11-04T03:33:04.6271572Z at
> akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235)
> 2021-11-04T03:33:04.6272125Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6272340Z
> 2021-11-04T03:33:04.6272798Z "process reaper" #8805 daemon prio=10 os_prio=0
> tid=0x00007f74000ae000 nid=0x16186 runnable [0x00007f7408051000]
> 2021-11-04T03:33:04.6273325Z java.lang.Thread.State: RUNNABLE
> 2021-11-04T03:33:04.6273750Z at
> java.lang.UNIXProcess.waitForProcessExit(Native Method)
> 2021-11-04T03:33:04.6274266Z at
> java.lang.UNIXProcess.lambda$initStreams$3(UNIXProcess.java:289)
> 2021-11-04T03:33:04.6274797Z at
> java.lang.UNIXProcess$$Lambda$1354/2086667208.run(Unknown Source)
> 2021-11-04T03:33:04.6295173Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> 2021-11-04T03:33:04.6295936Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.6296476Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6296707Z
> 2021-11-04T03:33:04.6297571Z "Thread-6" #18 daemon prio=5 os_prio=0
> tid=0x00007f73f4009000 nid=0x12e5d runnable [0x00007f740a21a000]
> 2021-11-04T03:33:04.6298114Z java.lang.Thread.State: RUNNABLE
> 2021-11-04T03:33:04.6298871Z at
> java.net.SocketInputStream.socketRead0(Native Method)
> 2021-11-04T03:33:04.6299394Z at
> java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
> 2021-11-04T03:33:04.6300127Z at
> java.net.SocketInputStream.read(SocketInputStream.java:171)
> 2021-11-04T03:33:04.6300796Z at
> java.net.SocketInputStream.read(SocketInputStream.java:141)
> 2021-11-04T03:33:04.6301625Z at
> sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
> 2021-11-04T03:33:04.6302471Z at
> sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
> 2021-11-04T03:33:04.6303169Z at
> sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
> 2021-11-04T03:33:04.6304217Z - locked <0x00000000936d68e0> (a
> java.io.InputStreamReader)
> 2021-11-04T03:33:04.6304748Z at
> java.io.InputStreamReader.read(InputStreamReader.java:184)
> 2021-11-04T03:33:04.6305267Z at
> java.io.BufferedReader.fill(BufferedReader.java:161)
> 2021-11-04T03:33:04.6305780Z at
> java.io.BufferedReader.readLine(BufferedReader.java:324)
> 2021-11-04T03:33:04.6306501Z - locked <0x00000000936d68e0> (a
> java.io.InputStreamReader)
> 2021-11-04T03:33:04.6307012Z at
> java.io.BufferedReader.readLine(BufferedReader.java:389)
> 2021-11-04T03:33:04.6307604Z at
> org.apache.flink.api.python.shaded.py4j.GatewayConnection.run(GatewayConnection.java:230)
> 2021-11-04T03:33:04.6308162Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6308366Z
> 2021-11-04T03:33:04.6309240Z "Thread-5" #17 daemon prio=5 os_prio=0
> tid=0x00007f73fc015000 nid=0x12e5b waiting on condition [0x00007f740a41c000]
> 2021-11-04T03:33:04.6310106Z java.lang.Thread.State: TIMED_WAITING
> (parking)
> 2021-11-04T03:33:04.6310476Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6311927Z - parking to wait for <0x00000000937cc9d0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6312695Z at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> 2021-11-04T03:33:04.6313338Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> 2021-11-04T03:33:04.6314293Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
> 2021-11-04T03:33:04.6315297Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> 2021-11-04T03:33:04.6316038Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.6316559Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.6317082Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.6317511Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6317810Z
> 2021-11-04T03:33:04.6318489Z "Thread-4" #15 daemon prio=5 os_prio=0
> tid=0x00007f73f4006800 nid=0x12e59 waiting on condition [0x00007f740a51d000]
> 2021-11-04T03:33:04.6318970Z java.lang.Thread.State: WAITING (parking)
> 2021-11-04T03:33:04.6319315Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6320095Z - parking to wait for <0x00000000e36dd710> (a
> java.util.concurrent.CompletableFuture$Signaller)
> 2021-11-04T03:33:04.6320583Z at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 2021-11-04T03:33:04.6321269Z at
> java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1707)
> 2021-11-04T03:33:04.6321788Z at
> java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
> 2021-11-04T03:33:04.6322291Z at
> java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1742)
> 2021-11-04T03:33:04.6322801Z at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
> 2021-11-04T03:33:04.6323519Z at
> org.apache.flink.table.api.internal.TableResultImpl.awaitInternal(TableResultImpl.java:123)
> 2021-11-04T03:33:04.6324081Z at
> org.apache.flink.table.api.internal.TableResultImpl.await(TableResultImpl.java:86)
> 2021-11-04T03:33:04.6324577Z at
> sun.reflect.GeneratedMethodAccessor209.invoke(Unknown Source)
> 2021-11-04T03:33:04.6325084Z at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2021-11-04T03:33:04.6325581Z at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2021-11-04T03:33:04.6326097Z at
> org.apache.flink.api.python.shaded.py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
> 2021-11-04T03:33:04.6326710Z at
> org.apache.flink.api.python.shaded.py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
> 2021-11-04T03:33:04.6328396Z at
> org.apache.flink.api.python.shaded.py4j.Gateway.invoke(Gateway.java:282)
> 2021-11-04T03:33:04.6329374Z at
> org.apache.flink.api.python.shaded.py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
> 2021-11-04T03:33:04.6330179Z at
> org.apache.flink.api.python.shaded.py4j.commands.CallCommand.execute(CallCommand.java:79)
> 2021-11-04T03:33:04.6331022Z at
> org.apache.flink.api.python.shaded.py4j.GatewayConnection.run(GatewayConnection.java:238)
> 2021-11-04T03:33:04.6331525Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6331726Z
> 2021-11-04T03:33:04.6332408Z "Thread-3" #14 daemon prio=5 os_prio=0
> tid=0x00007f73fc016000 nid=0x12e57 runnable [0x00007f740a31b000]
> 2021-11-04T03:33:04.6333048Z java.lang.Thread.State: RUNNABLE
> 2021-11-04T03:33:04.6333827Z at java.net.PlainSocketImpl.socketAccept(Native
> Method)
> 2021-11-04T03:33:04.6334291Z at
> java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
> 2021-11-04T03:33:04.6334782Z at
> java.net.ServerSocket.implAccept(ServerSocket.java:560)
> 2021-11-04T03:33:04.6335390Z at
> java.net.ServerSocket.accept(ServerSocket.java:528)
> 2021-11-04T03:33:04.6336054Z at
> org.apache.flink.api.python.shaded.py4j.GatewayServer.run(GatewayServer.java:685)
> 2021-11-04T03:33:04.6336530Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6336867Z
> 2021-11-04T03:33:04.6337712Z "Log4j2-TF-2-Scheduled-1" #9 daemon prio=5
> os_prio=0 tid=0x00007f7448586000 nid=0x12e53 waiting on condition
> [0x00007f740a81e000]
> 2021-11-04T03:33:04.6338463Z java.lang.Thread.State: TIMED_WAITING
> (parking)
> 2021-11-04T03:33:04.6338823Z at sun.misc.Unsafe.park(Native Method)
> 2021-11-04T03:33:04.6339551Z - parking to wait for <0x0000000093749450> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 2021-11-04T03:33:04.6340602Z at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> 2021-11-04T03:33:04.6341363Z at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> 2021-11-04T03:33:04.6342038Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
> 2021-11-04T03:33:04.6342689Z at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
> 2021-11-04T03:33:04.6343573Z at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
> 2021-11-04T03:33:04.6344280Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
> 2021-11-04T03:33:04.6344829Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2021-11-04T03:33:04.6345269Z at java.lang.Thread.run(Thread.java:748)
> 2021-11-04T03:33:04.6345467Z
> 2021-11-04T03:33:04.6345857Z "Service Thread" #7 daemon prio=9 os_prio=0
> tid=0x00007f74480be000 nid=0x12e51 runnable [0x0000000000000000]
> 2021-11-04T03:33:04.6346309Z java.lang.Thread.State: RUNNABLE
> 2021-11-04T03:33:04.6346631Z
> 2021-11-04T03:33:04.6347033Z "C1 CompilerThread1" #6 daemon prio=9 os_prio=0
> tid=0x00007f74480bb000 nid=0x12e50 waiting on condition [0x0000000000000000]
> 2021-11-04T03:33:04.6347641Z java.lang.Thread.State: RUNNABLE
> 2021-11-04T03:33:04.6347794Z
> 2021-11-04T03:33:04.6348183Z "C2 CompilerThread0" #5 daemon prio=9 os_prio=0
> tid=0x00007f74480b8000 nid=0x12e4f waiting on condition [0x0000000000000000]
> 2021-11-04T03:33:04.6348638Z java.lang.Thread.State: RUNNABLE
> 2021-11-04T03:33:04.6348788Z
> 2021-11-04T03:33:04.6349157Z "Signal Dispatcher" #4 daemon prio=9 os_prio=0
> tid=0x00007f74480b5000 nid=0x12e4e runnable [0x0000000000000000]
> 2021-11-04T03:33:04.6349593Z java.lang.Thread.State: RUNNABLE
> 2021-11-04T03:33:04.6349746Z
> 2021-11-04T03:33:04.6350115Z "Finalizer" #3 daemon prio=8 os_prio=0
> tid=0x00007f7448084800 nid=0x12e4d in Object.wait() [0x00007f74308ad000]
> 2021-11-04T03:33:04.6350578Z java.lang.Thread.State: WAITING (on object
> monitor)
> 2021-11-04T03:33:04.6350937Z at java.lang.Object.wait(Native Method)
> 2021-11-04T03:33:04.6351306Z at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
> 2021-11-04T03:33:04.6351956Z - locked <0x00000000936b9950> (a
> java.lang.ref.ReferenceQueue$Lock)
> 2021-11-04T03:33:04.6352396Z at
> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
> 2021-11-04T03:33:04.6353186Z at
> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)
> 2021-11-04T03:33:04.6353401Z
> 2021-11-04T03:33:04.6353787Z "Reference Handler" #2 daemon prio=10 os_prio=0
> tid=0x00007f7448080000 nid=0x12e4c in Object.wait() [0x00007f74309ae000]
> 2021-11-04T03:33:04.6354263Z java.lang.Thread.State: WAITING (on object
> monitor)
> 2021-11-04T03:33:04.6355482Z at java.lang.Object.wait(Native Method)
> 2021-11-04T03:33:04.6356457Z at java.lang.Object.wait(Object.java:502)
> 2021-11-04T03:33:04.6356954Z at
> java.lang.ref.Reference.tryHandlePending(Reference.java:191)
> 2021-11-04T03:33:04.6357793Z - locked <0x00000000937d9ba0> (a
> java.lang.ref.Reference$Lock)
> 2021-11-04T03:33:04.6358318Z at
> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
> 2021-11-04T03:33:04.6358603Z
> 2021-11-04T03:33:04.6360678Z "main" #1 prio=5 os_prio=0
> tid=0x00007f744800c800 nid=0x12e48 runnable [0x00007f7450c2f000]
> 2021-11-04T03:33:04.6361309Z java.lang.Thread.State: RUNNABLE
> 2021-11-04T03:33:04.6361697Z at
> java.net.SocketInputStream.socketRead0(Native Method)
> 2021-11-04T03:33:04.6362342Z at
> java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
> 2021-11-04T03:33:04.6363246Z at
> java.net.SocketInputStream.read(SocketInputStream.java:171)
> 2021-11-04T03:33:04.6369795Z at
> java.net.SocketInputStream.read(SocketInputStream.java:141)
> 2021-11-04T03:33:04.6370303Z at
> sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
> 2021-11-04T03:33:04.6370780Z at
> sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
> 2021-11-04T03:33:04.6371409Z at
> sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
> 2021-11-04T03:33:04.6372770Z - locked <0x00000000937dbd58> (a
> java.io.InputStreamReader)
> 2021-11-04T03:33:04.6373386Z at
> java.io.InputStreamReader.read(InputStreamReader.java:184)
> 2021-11-04T03:33:04.6373830Z at
> java.io.BufferedReader.fill(BufferedReader.java:161)
> 2021-11-04T03:33:04.6374268Z at
> java.io.BufferedReader.readLine(BufferedReader.java:324)
> 2021-11-04T03:33:04.6374864Z - locked <0x00000000937dbd58> (a
> java.io.InputStreamReader)
> 2021-11-04T03:33:04.6375504Z at
> java.io.BufferedReader.readLine(BufferedReader.java:389)
> 2021-11-04T03:33:04.6376569Z at
> org.apache.flink.api.python.shaded.py4j.CallbackConnection.readBlockingResponse(CallbackConnection.java:169)
> 2021-11-04T03:33:04.6377564Z at
> org.apache.flink.api.python.shaded.py4j.CallbackConnection.sendCommand(CallbackConnection.java:148)
> 2021-11-04T03:33:04.6378158Z at
> org.apache.flink.api.python.shaded.py4j.CallbackClient.sendCommand(CallbackClient.java:384)
> 2021-11-04T03:33:04.6378745Z at
> org.apache.flink.api.python.shaded.py4j.CallbackClient.sendCommand(CallbackClient.java:356)
> 2021-11-04T03:33:04.6379359Z at
> org.apache.flink.api.python.shaded.py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:106)
> 2021-11-04T03:33:04.6380018Z at com.sun.proxy.$Proxy19.ping(Unknown Source)
> 2021-11-04T03:33:04.6380615Z at
> org.apache.flink.client.python.PythonGatewayServer.main(PythonGatewayServer.java:94)
> 2021-11-04T03:33:04.6380867Z
> 2021-11-04T03:33:04.6381186Z "VM Thread" os_prio=0 tid=0x00007f7448076800
> nid=0x12e4b runnable
> 2021-11-04T03:33:04.6381390Z
> 2021-11-04T03:33:04.6381706Z "GC task thread#0 (ParallelGC)" os_prio=0
> tid=0x00007f7448021800 nid=0x12e49 runnable
> 2021-11-04T03:33:04.6381981Z
> 2021-11-04T03:33:04.6382318Z "GC task thread#1 (ParallelGC)" os_prio=0
> tid=0x00007f7448023800 nid=0x12e4a runnable
> 2021-11-04T03:33:04.6382529Z
> 2021-11-04T03:33:04.6382871Z "VM Periodic Task Thread" os_prio=0
> tid=0x00007f74480c0000 nid=0x12e52 waiting on condition
> 2021-11-04T03:33:04.6383092Z
> 2021-11-04T03:33:04.6383356Z JNI global references: 4529
> 2021-11-04T03:33:04.6383497Z
> 2021-11-04T03:33:04.6383814Z
> ==============================================================================
> 2021-11-04T03:33:04.6384200Z Printing stack trace of Java process 98730
> 2021-11-04T03:33:04.6384584Z
> ==============================================================================
> 2021-11-04T03:33:04.6385274Z Picked up JAVA_TOOL_OPTIONS:
> -XX:+HeapDumpOnOutOfMemoryError
> 2021-11-04T03:33:04.7205545Z 98730: No such process
> 2021-11-04T03:33:04.7375262Z Killing process with pid=600 and all descendants
> 2021-11-04T03:33:04.7873062Z /__w/1/s/tools/ci/watchdog.sh: line 113: 600
> Terminated $cmd
> 2021-11-04T03:33:04.7894908Z Process exited with EXIT CODE: 143.
> 2021-11-04T03:33:04.7895261Z Trying to KILL watchdog (595).
> 2021-11-04T03:33:04.7945265Z Searching for .dump, .dumpstream and related
> files in '/__w/1/s'
> 2021-11-04T03:33:10.8893307Z Compressing debug files
> 2021-11-04T03:33:10.8986582Z ./
> 2021-11-04T03:33:11.0641454Z ##[error]Bash exited with code '143'.
> 2021-11-04T03:33:11.3025139Z ##[section]Finishing: Test - python
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=25897&view=logs&j=821b528f-1eed-5598-a3b4-7f748b13f261&t=4fad9527-b9a5-5015-1b70-8356e5c91490&l=21416
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
