[ https://issues.apache.org/jira/browse/FLINK-28789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575360#comment-17575360 ]
Yingjie Cao commented on FLINK-28789: ------------------------------------- Though still not know the root cause, by reverting FLINK-28373 and testing multiple times on my own azure account, the issue seems resolved. For CI stability, I am reverting FLINK-28373, let's see if that solves the problem. > TPC-DS tests failed due to release input gate for task failure > ---------------------------------------------------------------- > > Key: FLINK-28789 > URL: https://issues.apache.org/jira/browse/FLINK-28789 > Project: Flink > Issue Type: Bug > Components: Runtime / Network > Affects Versions: 1.16.0 > Reporter: Leonard Xu > Assignee: Yuxin Tan > Priority: Blocker > Labels: test-stability > Fix For: 1.16.0 > > > {code:java} > switched from CANCELING to CANCELED. > 2022-08-03 08:03:02,776 INFO org.apache.flink.runtime.taskmanager.Task > [] - Freeing task resources for MultipleInput[2212] -> > Calc[2191] -> HashAggregate[2192] (8/8)#1 > (cf5f33b100f0efb21b9ff8d27a78cd8e_d806bb3f5ea308ac3f1df304a96163b4_7_1). > 2022-08-03 08:03:02,776 ERROR org.apache.flink.runtime.taskmanager.Task > [] - Failed to release input gate for task MultipleInput[2212] > -> Calc[2191] -> HashAggregate[2192] (8/8)#1. > org.apache.flink.shaded.netty4.io.netty.util.IllegalReferenceCountException: > refCnt: 0, decrement: 1 > at > org.apache.flink.shaded.netty4.io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74) > ~[flink-dist-1.16-SNAPSHOT.jar:1.16-SNAPSHOT] > at > org.apache.flink.shaded.netty4.io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138) > ~[flink-dist-1.16-SNAPSHOT.jar:1.16-SNAPSHOT] > at > org.apache.flink.shaded.netty4.io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100) > ~[flink-dist-1.16-SNAPSHOT.jar:1.16-SNAPSHOT] > at > org.apache.flink.runtime.io.network.buffer.NetworkBuffer.recycleBuffer(NetworkBuffer.java:156) > ~[flink-dist-1.16-SNAPSHOT.jar:1.16-SNAPSHOT] > at > org.apache.flink.runtime.io.network.buffer.ReadOnlySlicedNetworkBuffer.recycleBuffer(ReadOnlySlicedNetworkBuffer.java:123) > ~[flink-dist-1.16-SNAPSHOT.jar:1.16-SNAPSHOT] > at > org.apache.flink.runtime.io.network.buffer.CompositeBuffer.recycleBuffer(CompositeBuffer.java:70) > ~[flink-dist-1.16-SNAPSHOT.jar:1.16-SNAPSHOT] > at java.util.ArrayList.forEach(ArrayList.java:1259) ~[?:1.8.0_332] > at > org.apache.flink.runtime.io.network.partition.SortMergeSubpartitionReader.releaseInternal(SortMergeSubpartitionReader.java:181) > ~[flink-dist-1.16-SNAPSHOT.jar:1.16-SNAPSHOT] > at > org.apache.flink.runtime.io.network.partition.SortMergeSubpartitionReader.releaseAllResources(SortMergeSubpartitionReader.java:163) > ~[flink-dist-1.16-SNAPSHOT.jar:1.16-SNAPSHOT] > at > org.apache.flink.runtime.io.network.partition.consumer.LocalInputChannel.releaseAllResources(LocalInputChannel.java:341) > ~[flink-dist-1.16-SNAPSHOT.jar:1.16-SNAPSHOT] > at > org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate.close(SingleInputGate.java:667) > ~[flink-dist-1.16-SNAPSHOT.jar:1.16-SNAPSHOT] > at > org.apache.flink.runtime.taskmanager.InputGateWithMetrics.close(InputGateWithMetrics.java:140) > ~[flink-dist-1.16-SNAPSHOT.jar:1.16-SNAPSHOT] > at > org.apache.flink.runtime.taskmanager.Task.closeAllInputGates(Task.java:1010) > [flink-dist-1.16-SNAPSHOT.jar:1.16-SNAPSHOT] > at > org.apache.flink.runtime.taskmanager.Task.releaseResources(Task.java:975) > [flink-dist-1.16-SNAPSHOT.jar:1.16-SNAPSHOT] > at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:820) > [flink-dist-1.16-SNAPSHOT.jar:1.16-SNAPSHOT] > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550) > [flink-dist-1.16-SNAPSHOT.jar:1.16-SNAPSHOT] > at java.lang.Thread.run(Thread.java:750) [?:1.8.0_332] > 2022-08-03 08:03:02,778 WARN org.apache.flink.metrics.MetricGroup > {code} > The failed CI link: > https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=39152&view=results -- This message was sent by Atlassian Jira (v8.20.10#820010)