[
https://issues.apache.org/jira/browse/FLINK-17173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084921#comment-17084921
]
Jiahui Jiang commented on FLINK-17173:
--------------------------------------
Right now idleStateRetentionTime config is already on query level on table API
level. It seems to be hard to support finer granularity and set it per operator
there (since this is only a table API concept).
If they only want to set the retention time for some of the operators,
splitting up the query into several subqueries doesn't sound bad for usability
:D
> Supports query hint to config "IdleStateRetentionTime" per query in SQL
> -----------------------------------------------------------------------
>
> Key: FLINK-17173
> URL: https://issues.apache.org/jira/browse/FLINK-17173
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / API
> Affects Versions: 1.11.0
> Reporter: Danny Chen
> Priority: Major
>
> The motivation why we need this (copy from user mailing list [~qzhzm173227])
> In some of the use cases our users have, they have a couple of complex join
> queries where the key domains key evolving - we definitely want some sort of
> state retention for those queries; but there are other where the key domain
> doesn't evolve overtime, but there isn't really a guarantee on what's the
> maximum gap between 2 records of the same key to appear in the stream, we
> don't want to accidentally invalidate the state for those keys in these
> streams.
> Because of queries with different requirements can both exist in the
> pipeline, I think we have to config `IDLE_STATE_RETENTION_TIME` per operator.
> Just wondering, has similar requirement not come up much for SQL users
> before? (being able to set table / query configuration inside SQL queries)
> We are also a little bit concerned because right now since
> 'toRetractStream(Table, Class, QueryConfig)' is deprecated, relying on the
> fact that TableConfig is read during toDataStream feels like relying on an
> implementation details that just happens to work, and there is no guarantee
> that it will keep working in the future versions...
> Demo syntax:
> {code:sql}
> CREATE TABLE `/output` AS
> SELECT /*+ IDLE_STATE_RETENTION_TIME(minTime ='5m', maxTime ='11m') */ *
> FROM `/input1` a
> INNER JOIN `/input2` b
> ON a.column_name = b.column_name;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
