cpugputpu created FLINK-16387:
---------------------------------
Summary: Use LinkedHashMap for deterministic order in
HeapMapState.java
Key: FLINK-16387
URL: https://issues.apache.org/jira/browse/FLINK-16387
Project: Flink
Issue Type: Bug
Reporter: cpugputpu
The tests in
_org.apache.flink.table.runtime.operators.join.RowTimeBoundedStreamJoinTest#testRowTimeInnerJoinWithCommonBounds_
and
_org.apache.flink.table.runtime.operators.join.ProcTimeBoundedStreamJoinTest#testProcTimeInnerJoinWithCommonBounds_
fail due to a different iteration order of a HashMap. The failure is presented
as follows.
_org.junit.internal.ArrayComparisonFailure: output wrong.: arrays first
differed at element [0];
expected:<(+|1,1a1,1,1b3)>
but was:<(+|1,1a3,1,1b3)>
at
org.apache.flink.table.runtime.operators.join.ProcTimeBoundedStreamJoinTest.testProcTimeInnerJoinWithCommonBounds(ProcTimeBoundedStreamJoinTest.java:101_
The root cause of this failure lies in a HashMap's iterator, which makes no
guarantee about the iteration order. A brief stack trace is for your reference:
ava.util.HashMap$EntrySet.iterator(HashMap.java:1014)
org.apache.flink.runtime.state.heap.HeapMapState.iterator(HeapMapState.java:161)
org.apache.flink.runtime.state.UserFacingMapState.iterator(UserFacingMapState.java:95)
org.apache.flink.table.runtime.operators.join.TimeBoundedStreamJoin.processElement2(TimeBoundedStreamJoin.java:246)
org.apache.flink.table.runtime.operators.join.ProcTimeBoundedStreamJoin.processElement2(ProcTimeBoundedStreamJoin.java:29)
org.apache.flink.table.runtime.operators.join.TimeBoundedStreamJoin.processElement2(TimeBoundedStreamJoin.java:52)
org.apache.flink.streaming.api.operators.co.LegacyKeyedCoProcessOperator.processElement2(LegacyKeyedCoProcessOperator.java:89)
org.apache.flink.streaming.util.TwoInputStreamOperatorTestHarness.processElement2(TwoInputStreamOperatorTestHarness.java:57)
org.apache.flink.table.runtime.operators.join.ProcTimeBoundedStreamJoinTest.testProcTimeInnerJoinWithCommonBounds(ProcTimeBoundedStreamJoinTest.java:70)
The specification about HashMap says that "this class makes no guarantees as to
the order of the map; in particular, it does not guarantee that the order will
remain constant over time". The documentation is here for your reference:
[https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html]
The fix is to use LinkedHashMap instead of HashMap. In this way, the test will
not suffer from failure any more and the code will be more stable, free of this
non-deterministic behaviour.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
