[
https://issues.apache.org/jira/browse/FLINK-25217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17698295#comment-17698295
]
Timo Walther commented on FLINK-25217:
--------------------------------------
Thanks for offering your help [~jingge] and [~ConradJam]. Would be great if we
can split the work. Let me summarize the work that needs to be done:
We need to come up with a testing framework that contains input, pipeline,
output data. The pipeline should be defined by SQL (optionally Table API). The
input and output data should be split by a "before savepoint" and "after
restore" part. The annotations together with {{ExecNodeMetadataUtil}} will help
in test coverage. The pipeline definition should be usable across various tests.
I'm happy to sketch out the interfaces and their behavior. Then we can work
with joint forces on implementing pipelines for all operators.
What do you think? Do you have a particular timeline in mind?
> FLIP-190: Support Version Upgrades for Table API & SQL Programs
> ---------------------------------------------------------------
>
> Key: FLINK-25217
> URL: https://issues.apache.org/jira/browse/FLINK-25217
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API, Table SQL / Planner
> Reporter: Timo Walther
> Priority: Minor
> Labels: auto-deprioritized-major, pull-request-available
>
> Nowadays, the Table & SQL API is as important to Flink as the DataStream API.
> It is one of the main abstractions for expressing pipelines that perform
> stateful stream processing. Users expect the same backwards compatibility
> guarantees when upgrading to a newer Flink version as with the DataStream API.
> In particular, this means:
> * once the operator topology is defined, it remains static and does not
> change between Flink versions, unless resulting in better performance,
> * business logic (defined using expressions and functions in queries) behaves
> identical as before the version upgrade,
> * the state of a Table & SQL API program can be restored from a savepoint of
> a previous version,
> * adding or removing stateful operators should be made possible in the
> DataStream API.
> The same query can remain up and running after upgrades.
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=191336489
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
