[
https://issues.apache.org/jira/browse/FLINK-18540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157838#comment-17157838
]
WeiqiangSun commented on FLINK-18540:
-------------------------------------
[~libenchao] Mini batch may not work well with retract messages for HBase
connector. Unnecessary retract messages can cause puts and deletes with same
timestamp. As for HBase, deletes can mask puts with the same timestamp.
https://issues.apache.org/jira/browse/HBASE-2256
https://issues.apache.org/jira/browse/HBASE-8721
> Unnecessary retract messages when setIdleStateRetentionTime before converting
> dynamic table to retract stream
> -------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-18540
> URL: https://issues.apache.org/jira/browse/FLINK-18540
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner, Table SQL / Runtime
> Affects Versions: 1.10.0
> Reporter: WeiqiangSun
> Priority: Critical
>
> Unnecessary retract messages when setIdleStateRetentionTime before converting
> dynamic table to retract stream。
>
> {code:java}
> final StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment();
> env.setParallelism(1);
> EnvironmentSettings es =
> EnvironmentSettings.newInstance().inStreamingMode().build();
> StreamTableEnvironment fsTableEnv = StreamTableEnvironment.create(env,es);
> DataStream<Integer> ds = env.fromElements(5,4,3,2,1);
> // if do not setIdleStateRetentionTime, it will output only one message as
> expected like '(true,5)'
> // if do setIdleStateRetentionTime, it will output unnecessary retract
> messages like '(false,5)/(true,5)'
> fsTableEnv.getConfig().setIdleStateRetentionTime(Time.hours(1000000),Time.hours(10000000));
> fsTableEnv.createTemporaryView("test",ds,"id");
> Table test = fsTableEnv.sqlQuery("select max(id) from test");
> fsTableEnv.toRetractStream(test,Row.class).printToErr();
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
