[ https://issues.apache.org/jira/browse/FLINK-22002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17405103#comment-17405103 ]
Till Rohrmann commented on FLINK-22002: --------------------------------------- The test failure has a different cause {code} Aug 26 00:51:29 [ERROR] testSingleAggOnTable_HashAgg_WithoutLocalAgg Time elapsed: 29.979 s <<< ERROR! Aug 26 00:51:29 java.lang.RuntimeException: Failed to fetch next result Aug 26 00:51:29 at org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:109) Aug 26 00:51:29 at org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:80) Aug 26 00:51:29 at org.apache.flink.table.api.internal.TableResultImpl$CloseableRowIteratorWrapper.hasNext(TableResultImpl.java:370) Aug 26 00:51:29 at java.util.Iterator.forEachRemaining(Iterator.java:115) Aug 26 00:51:29 at org.apache.flink.util.CollectionUtil.iteratorToList(CollectionUtil.java:109) Aug 26 00:51:29 at org.apache.flink.table.planner.runtime.utils.BatchTestBase.executeQuery(BatchTestBase.scala:300) Aug 26 00:51:29 at org.apache.flink.table.planner.runtime.utils.BatchTestBase.check(BatchTestBase.scala:140) Aug 26 00:51:29 at org.apache.flink.table.planner.runtime.utils.BatchTestBase.checkResult(BatchTestBase.scala:106) Aug 26 00:51:29 at org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase.testSingleAggOnTable(AggregateReduceGroupingITCase.scala:179) Aug 26 00:51:29 at org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase.testSingleAggOnTable_HashAgg_WithoutLocalAgg(AggregateReduceGroupingITCase.scala:143) Aug 26 00:51:29 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Aug 26 00:51:29 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) Aug 26 00:51:29 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Aug 26 00:51:29 at java.lang.reflect.Method.invoke(Method.java:498) Aug 26 00:51:29 at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) Aug 26 00:51:29 at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) Aug 26 00:51:29 at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) Aug 26 00:51:29 at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) Aug 26 00:51:29 at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) Aug 26 00:51:29 at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) Aug 26 00:51:29 at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) Aug 26 00:51:29 at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) Aug 26 00:51:29 at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) Aug 26 00:51:29 at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) Aug 26 00:51:29 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) Aug 26 00:51:29 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) Aug 26 00:51:29 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) Aug 26 00:51:29 at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) Aug 26 00:51:29 at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) Aug 26 00:51:29 at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) Aug 26 00:51:29 at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) Aug 26 00:51:29 at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) Aug 26 00:51:29 at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) Aug 26 00:51:29 at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) Aug 26 00:51:29 at org.junit.rules.RunRules.evaluate(RunRules.java:20) Aug 26 00:51:29 at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) {code} Looking at the logs we do see the following: {code} 00:50:40,740 [flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Offer reserved slots to the leader of job e3bd09da12185c0e070507670d2939ee. 00:50:40,740 [flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> SourceConversion(table=[default_catalog.default_database.T6], fields=[a6, b6, c6, d6, e6, f6]) -> Calc(select=[a6, d6, b6, c6, e6]) (1/1) (a4a17596e5831da44e9ba35e8df94a05) switched from SCHEDULED to DEPLOYING. 00:50:40,740 [flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source: Custom Source -> SourceConversion(table=[default_catalog.default_database.T6], fields=[a6, b6, c6, d6, e6, f6]) -> Calc(select=[a6, d6, b6, c6, e6]) (1/1) (attempt #0) with attempt id a4a17596e5831da44e9ba35e8df94a05 to 02c21b4a-d30f-47ea-a3d6-32f3cfbf4986 @ localhost (dataPort=-1) with allocation id a7ce50d9a16d942218f87aa0cf8e4417 00:51:01,788 [flink-akka.actor.default-dispatcher-9] INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:1, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=341.333gb (366503875925 bytes), taskOffHeapMemory=341.333gb (366503875925 bytes), managedMemory=33.333mb (34952533 bytes), networkMemory=21.333mb (22369621 bytes)}, allocationId: a7ce50d9a16d942218f87aa0cf8e4417, jobId: e3bd09da12185c0e070507670d2939ee). 00:51:01,791 [flink-akka.actor.default-dispatcher-9] INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:2, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=341.333gb (366503875925 bytes), taskOffHeapMemory=341.333gb (366503875925 bytes), managedMemory=33.333mb (34952533 bytes), networkMemory=21.333mb (22369621 bytes)}, allocationId: e9079548bfe4853e6250c98b7f1e6371, jobId: e3bd09da12185c0e070507670d2939ee). 00:51:01,792 [flink-akka.actor.default-dispatcher-9] INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:0, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=341.333gb (366503875925 bytes), taskOffHeapMemory=341.333gb (366503875925 bytes), managedMemory=33.333mb (34952533 bytes), networkMemory=21.333mb (22369621 bytes)}, allocationId: 95db7e01f53ed34c5792bb9e43038c7a, jobId: e3bd09da12185c0e070507670d2939ee). 00:51:01,792 [flink-akka.actor.default-dispatcher-9] INFO org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService [] - Remove job e3bd09da12185c0e070507670d2939ee from job leader monitoring. 00:51:01,792 [flink-akka.actor.default-dispatcher-9] INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Close JobManager connection for job e3bd09da12185c0e070507670d2939ee. 00:51:01,799 [flink-akka.actor.default-dispatcher-9] INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Receive slot request b170048e627e24093fce8580c71394b8 for job e3bd09da12185c0e070507670d2939ee from resource manager with leader id a65351795d762653d778817b27014477. 00:51:01,800 [flink-akka.actor.default-dispatcher-9] INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Allocated slot for b170048e627e24093fce8580c71394b8. 00:51:01,800 [flink-akka.actor.default-dispatcher-9] INFO org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService [] - Add job e3bd09da12185c0e070507670d2939ee for job leader monitoring. 00:51:01,801 [flink-akka.actor.default-dispatcher-9] INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Receive slot request 3832cf65f8a9cc21cb9398157dc67f24 for job e3bd09da12185c0e070507670d2939ee from resource manager with leader id a65351795d762653d778817b27014477. 00:51:01,801 [flink-akka.actor.default-dispatcher-9] INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Allocated slot for 3832cf65f8a9cc21cb9398157dc67f24. 00:51:01,801 [mini-cluster-io-thread-4] INFO org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService [] - Try to register at job manager akka://flink/user/rpc/jobmanager_1861 with leader id 69deb619-c222-4886-b6b7-5b06cc6a43d2. 00:51:01,801 [flink-akka.actor.default-dispatcher-9] INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Receive slot request 86336a918b4c450823a843b75bc61821 for job e3bd09da12185c0e070507670d2939ee from resource manager with leader id a65351795d762653d778817b27014477. 00:51:01,801 [flink-akka.actor.default-dispatcher-9] INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Allocated slot for 86336a918b4c450823a843b75bc61821. 00:51:01,816 [flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - HashAggregate(isMerge=[false], groupBy=[a6], auxGrouping=[d6], select=[a6, d6, AVG(b6) AS EXPR$2, COUNT(c6) AS EXPR$3, AVG(e6) AS EXPR$4]) -> NotNullEnforcer(fields=[EXPR$3]) (1/3) (33cc657a39f6b9d17bd8d54d2124a626) switched from SCHEDULED to DEPLOYING. 00:51:01,816 [flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying HashAggregate(isMerge=[false], groupBy=[a6], auxGrouping=[d6], select=[a6, d6, AVG(b6) AS EXPR$2, COUNT(c6) AS EXPR$3, AVG(e6) AS EXPR$4]) -> NotNullEnforcer(fields=[EXPR$3]) (1/3) (attempt #0) with attempt id 33cc657a39f6b9d17bd8d54d2124a626 to 02c21b4a-d30f-47ea-a3d6-32f3cfbf4986 @ localhost (dataPort=-1) with allocation id a7ce50d9a16d942218f87aa0cf8e4417 00:51:01,821 [flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - HashAggregate(isMerge=[false], groupBy=[a6], auxGrouping=[d6], select=[a6, d6, AVG(b6) AS EXPR$2, COUNT(c6) AS EXPR$3, AVG(e6) AS EXPR$4]) -> NotNullEnforcer(fields=[EXPR$3]) (2/3) (837dd9e3c0f15f5681a98f0104c1386d) switched from SCHEDULED to DEPLOYING. 00:51:01,821 [flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying HashAggregate(isMerge=[false], groupBy=[a6], auxGrouping=[d6], select=[a6, d6, AVG(b6) AS EXPR$2, COUNT(c6) AS EXPR$3, AVG(e6) AS EXPR$4]) -> NotNullEnforcer(fields=[EXPR$3]) (2/3) (attempt #0) with attempt id 837dd9e3c0f15f5681a98f0104c1386d to 02c21b4a-d30f-47ea-a3d6-32f3cfbf4986 @ localhost (dataPort=-1) with allocation id 95db7e01f53ed34c5792bb9e43038c7a 00:51:01,822 [flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - HashAggregate(isMerge=[false], groupBy=[a6], auxGrouping=[d6], select=[a6, d6, AVG(b6) AS EXPR$2, COUNT(c6) AS EXPR$3, AVG(e6) AS EXPR$4]) -> NotNullEnforcer(fields=[EXPR$3]) (3/3) (65295ad2b4be734a6685edb56ac5e883) switched from SCHEDULED to DEPLOYING. 00:51:01,822 [flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying HashAggregate(isMerge=[false], groupBy=[a6], auxGrouping=[d6], select=[a6, d6, AVG(b6) AS EXPR$2, COUNT(c6) AS EXPR$3, AVG(e6) AS EXPR$4]) -> NotNullEnforcer(fields=[EXPR$3]) (3/3) (attempt #0) with attempt id 65295ad2b4be734a6685edb56ac5e883 to 02c21b4a-d30f-47ea-a3d6-32f3cfbf4986 @ localhost (dataPort=-1) with allocation id e9079548bfe4853e6250c98b7f1e6371 00:51:01,822 [flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Sink: Collect table sink (1/1) (6338999cdc530bb3a2528b2501a473df) switched from SCHEDULED to DEPLOYING. 00:51:01,822 [flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Sink: Collect table sink (1/1) (attempt #0) with attempt id 6338999cdc530bb3a2528b2501a473df to 02c21b4a-d30f-47ea-a3d6-32f3cfbf4986 @ localhost (dataPort=-1) with allocation id a7ce50d9a16d942218f87aa0cf8e4417 00:51:02,134 [flink-akka.actor.default-dispatcher-10] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - HashAggregate(isMerge=[false], groupBy=[a6], auxGrouping=[d6], select=[a6, d6, AVG(b6) AS EXPR$2, COUNT(c6) AS EXPR$3, AVG(e6) AS EXPR$4]) -> NotNullEnforcer(fields=[EXPR$3]) (3/3) (65295ad2b4be734a6685edb56ac5e883) switched from DEPLOYING to FAILED on 02c21b4a-d30f-47ea-a3d6-32f3cfbf4986 @ localhost (dataPort=-1). org.apache.flink.util.FlinkException: TaskExecutor akka://flink/user/rpc/taskmanager_1815 has no more allocated slots for job e3bd09da12185c0e070507670d2939ee. at org.apache.flink.runtime.taskexecutor.TaskExecutor.closeJobManagerConnectionIfNoAllocatedResources(TaskExecutor.java:1936) ~[flink-runtime-1.14-SNAPSHOT.jar:1.14-SNAPSHOT] at org.apache.flink.runtime.taskexecutor.TaskExecutor.freeSlotInternal(TaskExecutor.java:1917) ~[flink-runtime-1.14-SNAPSHOT.jar:1.14-SNAPSHOT] at org.apache.flink.runtime.taskexecutor.TaskExecutor.timeoutSlot(TaskExecutor.java:1950) ~[flink-runtime-1.14-SNAPSHOT.jar:1.14-SNAPSHOT] at org.apache.flink.runtime.taskexecutor.TaskExecutor.access$3200(TaskExecutor.java:183) ~[flink-runtime-1.14-SNAPSHOT.jar:1.14-SNAPSHOT] at org.apache.flink.runtime.taskexecutor.TaskExecutor$SlotActionsImpl.lambda$timeoutSlot$1(TaskExecutor.java:2352) ~[flink-runtime-1.14-SNAPSHOT.jar:1.14-SNAPSHOT] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRunAsync$4(AkkaRpcActor.java:455) ~[flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68) ~[flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:455) ~[flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:213) ~[flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163) ~[flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) [flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) [flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) [flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) [flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) [flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at akka.actor.Actor.aroundReceive(Actor.scala:537) [flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at akka.actor.Actor.aroundReceive$(Actor.scala:535) [flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) [flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) [flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at akka.actor.ActorCell.invoke(ActorCell.scala:548) [flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) [flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at akka.dispatch.Mailbox.run(Mailbox.scala:231) [flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at akka.dispatch.Mailbox.exec(Mailbox.scala:243) [flink-rpc-akka_9779ccfd-a37e-4ef5-bfd6-6a4f3a80243c.jar:1.14-SNAPSHOT] at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) [?:1.8.0_292] at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) [?:1.8.0_292] at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) [?:1.8.0_292] at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) [?:1.8.0_292] {code} There is a 20s gap between offering the slots and accepting them. Due to this, I suspect that we run into the {{taskmanager.slot.timeout}} that defaults to 10s. With FLINK-21428, we have introduced this option and decoupled if from the {{akka.ask.timeout}}. I think it would be better if this option falls back to {{akka.ask.timeout}} if not explicitly set. That way, all tests will benefit from FLINK-23906 that increases the {{akka.ask.timeout}} to 5 minutes. > AggregateReduceGroupingITCase.testSingleAggOnTable_HashAgg_WithLocalAgg fail > because of submitting task time-out. > ----------------------------------------------------------------------------------------------------------------- > > Key: FLINK-22002 > URL: https://issues.apache.org/jira/browse/FLINK-22002 > Project: Flink > Issue Type: Bug > Components: Runtime / Coordination > Affects Versions: 1.12.2, 1.14.0 > Reporter: Guowei Ma > Priority: Major > Labels: test-stability > Fix For: 1.14.0 > > > https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=15634&view=logs&j=955770d3-1fed-5a0a-3db6-0c7554c910cb&t=14447d61-56b4-5000-80c1-daa459247f6a&l=6424 > {code:java} > org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase > 2021-03-29T00:27:25.3406344Z [ERROR] > testSingleAggOnTable_HashAgg_WithLocalAgg(org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase) > Time elapsed: 21.908 s <<< ERROR! > 2021-03-29T00:27:25.3407190Z java.lang.RuntimeException: Failed to fetch next > result > 2021-03-29T00:27:25.3407792Z at > org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:109) > 2021-03-29T00:27:25.3408502Z at > org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:80) > 2021-03-29T00:27:25.3409188Z at > org.apache.flink.table.planner.sinks.SelectTableSinkBase$RowIteratorWrapper.hasNext(SelectTableSinkBase.java:117) > 2021-03-29T00:27:25.3416724Z at > org.apache.flink.table.api.internal.TableResultImpl$CloseableRowIteratorWrapper.hasNext(TableResultImpl.java:350) > 2021-03-29T00:27:25.3417510Z at > java.util.Iterator.forEachRemaining(Iterator.java:115) > 2021-03-29T00:27:25.3418416Z at > org.apache.flink.util.CollectionUtil.iteratorToList(CollectionUtil.java:108) > 2021-03-29T00:27:25.3419031Z at > org.apache.flink.table.planner.runtime.utils.BatchTestBase.executeQuery(BatchTestBase.scala:298) > 2021-03-29T00:27:25.3419657Z at > org.apache.flink.table.planner.runtime.utils.BatchTestBase.check(BatchTestBase.scala:138) > 2021-03-29T00:27:25.3420638Z at > org.apache.flink.table.planner.runtime.utils.BatchTestBase.checkResult(BatchTestBase.scala:104) > 2021-03-29T00:27:25.3421384Z at > org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase.testSingleAggOnTable(AggregateReduceGroupingITCase.scala:182) > 2021-03-29T00:27:25.3422284Z at > org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase.testSingleAggOnTable_HashAgg_WithLocalAgg(AggregateReduceGroupingITCase.scala:135) > 2021-03-29T00:27:25.3422975Z at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > 2021-03-29T00:27:25.3423504Z at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > 2021-03-29T00:27:25.3424298Z at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > 2021-03-29T00:27:25.3425229Z at > java.lang.reflect.Method.invoke(Method.java:498) > 2021-03-29T00:27:25.3426107Z at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > 2021-03-29T00:27:25.3426756Z at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > 2021-03-29T00:27:25.3427743Z at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > 2021-03-29T00:27:25.3428520Z at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > 2021-03-29T00:27:25.3429128Z at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > 2021-03-29T00:27:25.3429715Z at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > 2021-03-29T00:27:25.3433435Z at > org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) > 2021-03-29T00:27:25.3433977Z at > org.junit.rules.RunRules.evaluate(RunRules.java:20) > 2021-03-29T00:27:25.3434476Z at > org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) > 2021-03-29T00:27:25.3435607Z at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) > 2021-03-29T00:27:25.3436460Z at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) > 2021-03-29T00:27:25.3437054Z at > org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > 2021-03-29T00:27:25.3437673Z at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > 2021-03-29T00:27:25.3438765Z at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > 2021-03-29T00:27:25.3439362Z at > org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > 2021-03-29T00:27:25.3440504Z at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > 2021-03-29T00:27:25.3441100Z at > org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) > 2021-03-29T00:27:25.3441673Z at > org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) > 2021-03-29T00:27:25.3442205Z at > org.junit.rules.RunRules.evaluate(RunRules.java:20) > 2021-03-29T00:27:25.3442710Z at > org.junit.runners.ParentRunner.run(ParentRunner.java:363) > 2021-03-29T00:27:25.3443420Z at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) > 2021-03-29T00:27:25.3444095Z at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) > 2021-03-29T00:27:25.3444749Z at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) > 2021-03-29T00:27:25.3445380Z at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) > 2021-03-29T00:27:25.3446224Z at > org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) > 2021-03-29T00:27:25.3447054Z at > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) > 2021-03-29T00:27:25.3447698Z at > org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) > 2021-03-29T00:27:25.3448295Z at > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) > 2021-03-29T00:27:25.3448851Z Caused by: java.io.IOException: Failed to fetch > job execution result > 2021-03-29T00:27:25.3449667Z at > org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.getAccumulatorResults(CollectResultFetcher.java:169) > 2021-03-29T00:27:25.3450406Z at > org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.next(CollectResultFetcher.java:118) > 2021-03-29T00:27:25.3451138Z at > org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:106) > 2021-03-29T00:27:25.3451674Z ... 42 more > 2021-03-29T00:27:25.3452201Z Caused by: > java.util.concurrent.ExecutionException: > org.apache.flink.runtime.client.JobExecutionException: Job execution failed. > 2021-03-29T00:27:25.3452872Z at > java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) > 2021-03-29T00:27:25.3453864Z at > java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928) > 2021-03-29T00:27:25.3454600Z at > org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.getAccumulatorResults(CollectResultFetcher.java:167) > 2021-03-29T00:27:25.3455163Z ... 44 more > 2021-03-29T00:27:25.3455633Z Caused by: > org.apache.flink.runtime.client.JobExecutionException: Job execution failed. > 2021-03-29T00:27:25.3456722Z at > org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144) > 2021-03-29T00:27:25.3457420Z at > org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$2(MiniClusterJobClient.java:117) > 2021-03-29T00:27:25.3458088Z at > java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616) > 2021-03-29T00:27:25.3458672Z at > java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:628) > 2021-03-29T00:27:25.3459380Z at > java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:1996) > 2021-03-29T00:27:25.3460223Z at > org.apache.flink.runtime.minicluster.MiniClusterJobClient.getJobExecutionResult(MiniClusterJobClient.java:114) > 2021-03-29T00:27:25.3460987Z at > org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.getAccumulatorResults(CollectResultFetcher.java:166) > 2021-03-29T00:27:25.3461530Z ... 44 more > 2021-03-29T00:27:25.3462007Z Caused by: > org.apache.flink.runtime.JobException: Recovery is suppressed by > NoRestartBackoffTimeStrategy > 2021-03-29T00:27:25.3462740Z at > org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:118) > 2021-03-29T00:27:25.3463566Z at > org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:80) > 2021-03-29T00:27:25.3464342Z at > org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:233) > 2021-03-29T00:27:25.3465041Z at > org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:224) > 2021-03-29T00:27:25.3465873Z at > org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:215) > 2021-03-29T00:27:25.3466611Z at > org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:669) > 2021-03-29T00:27:25.3467405Z at > org.apache.flink.runtime.scheduler.UpdateSchedulerNgOnInternalFailuresListener.notifyTaskFailure(UpdateSchedulerNgOnInternalFailuresListener.java:56) > 2021-03-29T00:27:25.3468253Z at > org.apache.flink.runtime.executiongraph.ExecutionGraph.notifySchedulerNgAboutInternalTaskFailure(ExecutionGraph.java:1869) > 2021-03-29T00:27:25.3469061Z at > org.apache.flink.runtime.executiongraph.Execution.processFail(Execution.java:1437) > 2021-03-29T00:27:25.3469687Z at > org.apache.flink.runtime.executiongraph.Execution.processFail(Execution.java:1377) > 2021-03-29T00:27:25.3470309Z at > org.apache.flink.runtime.executiongraph.Execution.markFailed(Execution.java:1205) > 2021-03-29T00:27:25.3471109Z at > org.apache.flink.runtime.executiongraph.Execution.lambda$deploy$11(Execution.java:856) > 2021-03-29T00:27:25.3471720Z at > java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) > 2021-03-29T00:27:25.3472333Z at > java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) > 2021-03-29T00:27:25.3472927Z at > java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456) > 2021-03-29T00:27:25.3473530Z at > org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:440) > 2021-03-29T00:27:25.3474147Z at > org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:208) > 2021-03-29T00:27:25.3474801Z at > org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77) > 2021-03-29T00:27:25.3475437Z at > org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158) > 2021-03-29T00:27:25.3476005Z at > akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) > 2021-03-29T00:27:25.3476522Z at > akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) > 2021-03-29T00:27:25.3477047Z at > scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) > 2021-03-29T00:27:25.3477587Z at > akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) > 2021-03-29T00:27:25.3478127Z at > scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) > 2021-03-29T00:27:25.3478663Z at > scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) > 2021-03-29T00:27:25.3479199Z at > scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) > 2021-03-29T00:27:25.3479964Z at > akka.actor.Actor$class.aroundReceive(Actor.scala:517) > 2021-03-29T00:27:25.3481778Z at > akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) > 2021-03-29T00:27:25.3482443Z at > akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) > 2021-03-29T00:27:25.3483152Z at > akka.actor.ActorCell.invoke(ActorCell.scala:561) > 2021-03-29T00:27:25.3483668Z at > akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) > 2021-03-29T00:27:25.3484339Z at akka.dispatch.Mailbox.run(Mailbox.scala:225) > 2021-03-29T00:27:25.3484999Z at akka.dispatch.Mailbox.exec(Mailbox.scala:235) > 2021-03-29T00:27:25.3485922Z at > akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) > 2021-03-29T00:27:25.3486533Z at > akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) > 2021-03-29T00:27:25.3487139Z at > akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > 2021-03-29T00:27:25.3487750Z at > akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) > 2021-03-29T00:27:25.3488903Z Caused by: > java.util.concurrent.CompletionException: > java.util.concurrent.TimeoutException: Invocation of public abstract > java.util.concurrent.CompletableFuture > org.apache.flink.runtime.taskexecutor.TaskExecutorGateway.submitTask(org.apache.flink.runtime.deployment.TaskDeploymentDescriptor,org.apache.flink.runtime.jobmaster.JobMasterId,org.apache.flink.api.common.time.Time) > timed out. > 2021-03-29T00:27:25.3490086Z at > java.util.concurrent.CompletableFuture.encodeRelay(CompletableFuture.java:326) > 2021-03-29T00:27:25.3490729Z at > java.util.concurrent.CompletableFuture.completeRelay(CompletableFuture.java:338) > 2021-03-29T00:27:25.3491369Z at > java.util.concurrent.CompletableFuture.uniRelay(CompletableFuture.java:925) > 2021-03-29T00:27:25.3492002Z at > java.util.concurrent.CompletableFuture$UniRelay.tryFire(CompletableFuture.java:913) > 2021-03-29T00:27:25.3492646Z at > java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) > 2021-03-29T00:27:25.3493299Z at > java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) > 2021-03-29T00:27:25.3494129Z at > org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:234) > 2021-03-29T00:27:25.3494833Z at > java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) > 2021-03-29T00:27:25.3495506Z at > java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) > 2021-03-29T00:27:25.3496159Z at > java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) > 2021-03-29T00:27:25.3496816Z at > java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) > 2021-03-29T00:27:25.3497477Z at > org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:1044) > 2021-03-29T00:27:25.3498061Z at > akka.dispatch.OnComplete.internal(Future.scala:263) > 2021-03-29T00:27:25.3498569Z at > akka.dispatch.OnComplete.internal(Future.scala:261) > 2021-03-29T00:27:25.3499094Z at > akka.dispatch.japi$CallbackBridge.apply(Future.scala:191) > 2021-03-29T00:27:25.3499642Z at > akka.dispatch.japi$CallbackBridge.apply(Future.scala:188) > 2021-03-29T00:27:25.3500344Z at > scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36) > 2021-03-29T00:27:25.3501155Z at > org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:73) > 2021-03-29T00:27:25.3502325Z at > scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44) > 2021-03-29T00:27:25.3503122Z at > scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252) > 2021-03-29T00:27:25.3503739Z at > akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:644) > 2021-03-29T00:27:25.3504306Z at > akka.actor.Scheduler$$anon$4.run(Scheduler.scala:205) > 2021-03-29T00:27:25.3504894Z at > scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601) > 2021-03-29T00:27:25.3505528Z at > scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109) > 2021-03-29T00:27:25.3506142Z at > scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599) > 2021-03-29T00:27:25.3506803Z at > akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328) > 2021-03-29T00:27:25.3507875Z at > akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279) > 2021-03-29T00:27:25.3509196Z at > akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283) > 2021-03-29T00:27:25.3510052Z at > akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235) > 2021-03-29T00:27:25.3510597Z at java.lang.Thread.run(Thread.java:748) > 2021-03-29T00:27:25.3511727Z Caused by: > java.util.concurrent.TimeoutException: Invocation of public abstract > java.util.concurrent.CompletableFuture > org.apache.flink.runtime.taskexecutor.TaskExecutorGateway.submitTask(org.apache.flink.runtime.deployment.TaskDeploymentDescriptor,org.apache.flink.runtime.jobmaster.JobMasterId,org.apache.flink.api.common.time.Time) > timed out. > 2021-03-29T00:27:25.3512850Z at > org.apache.flink.runtime.jobmaster.RpcTaskManagerGateway.submitTask(RpcTaskManagerGateway.java:68) > 2021-03-29T00:27:25.3513557Z at > org.apache.flink.runtime.executiongraph.Execution.lambda$deploy$10(Execution.java:832) > 2021-03-29T00:27:25.3514225Z at > java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604) > 2021-03-29T00:27:25.3514854Z at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > 2021-03-29T00:27:25.3515424Z at > java.util.concurrent.FutureTask.run(FutureTask.java:266) > 2021-03-29T00:27:25.3516090Z at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > 2021-03-29T00:27:25.3516881Z at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > 2021-03-29T00:27:25.3517585Z at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > 2021-03-29T00:27:25.3518218Z at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > 2021-03-29T00:27:25.3518794Z ... 1 more > 2021-03-29T00:27:25.3547493Z Caused by: akka.pattern.AskTimeoutException: Ask > timed out on [Actor[akka://flink/user/rpc/taskmanager_797#796750252]] after > [10000 ms]. Message of type > [org.apache.flink.runtime.rpc.messages.LocalRpcInvocation]. A typical reason > for `AskTimeoutException` is that the recipient actor didn't send a reply. > 2021-03-29T00:27:25.3548645Z at > akka.pattern.PromiseActorRef$$anonfun$2.apply(AskSupport.scala:635) > 2021-03-29T00:27:25.3549245Z at > akka.pattern.PromiseActorRef$$anonfun$2.apply(AskSupport.scala:635) > 2021-03-29T00:27:25.3550046Z at > akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:648) > 2021-03-29T00:27:25.3550605Z at > akka.actor.Scheduler$$anon$4.run(Scheduler.scala:205) > 2021-03-29T00:27:25.3551179Z at > scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601) > 2021-03-29T00:27:25.3551798Z at > scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109) > 2021-03-29T00:27:25.3552404Z at > scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599) > 2021-03-29T00:27:25.3553050Z at > akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328) > 2021-03-29T00:27:25.3554300Z at > akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279) > 2021-03-29T00:27:25.3555232Z at > akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283) > 2021-03-29T00:27:25.3555928Z at > akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235) > 2021-03-29T00:27:25.3556425Z ... 1 more > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)