Hi Benchao,
this is a very good question. I will update the FLIP about this.
The legacy planner will not support the new interfaces. It will only
support the old interfaces. With the next release, I think the Blink
planner is stable enough to be the default one as well.
Regards,
Timo
On 18.03.20 08:45, Benchao Li wrote:
Hi Timo,
Thank you and others for the efforts to prepare this FLIP.
The FLIP LGTM generally.
+1 for moving blink data structures to table-common, it's useful to udf too
in the future.
A little question is, do we plan to support the new interfaces and data
types in legacy planner?
Or we only plan to support these new interfaces in blink planner.
And using primary keys from DDL instead of derived key information from
each query is also a good idea,
we met some use cases where this does not works very well before.
This FLIP also makes the dependencies of table modules more clear, I like
it very much.
Timo Walther <twal...@apache.org> 于2020年3月17日周二 上午1:36写道：
Hi everyone,
I'm happy to present the results of long discussions that we had
internally. Jark, Dawid, Aljoscha, Kurt, Jingsong, me, and many more
have contributed to this design document.
We would like to propose new long-term table source and table sink
interfaces:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-95%3A+New+TableSource+and+TableSink+interfaces
This is a requirement for FLIP-105 and finalizing FLIP-32.
The goals of this FLIP are:
- Simplify the current interface architecture:
- Merge upsert, retract, and append sinks.
- Unify batch and streaming sources.
- Unify batch and streaming sinks.
- Allow sources to produce a changelog:
- UpsertTableSources have been requested a lot by users. Now is the
time to open the internal planner capabilities via the new interfaces.
- According to FLIP-105, we would like to support changelogs for
processing formats such as Debezium.
- Don't rely on DataStream API for source and sinks:
- According to FLIP-32, the Table API and SQL should be independent
of the DataStream API which is why the `table-common` module has no
dependencies on `flink-streaming-java`.
- Source and sink implementations should only depend on the
`table-common` module after FLIP-27.
- Until FLIP-27 is ready, we still put most of the interfaces in
`table-common` and strictly separate interfaces that communicate with a
planner and actual runtime reader/writers.
- Implement efficient sources and sinks without planner dependencies:
- Make Blink's internal data structures available to connectors.
- Introduce stable interfaces for data structures that can be
marked as `@PublicEvolving`.
- Only require dependencies on `flink-table-common` in the future
It finalizes the concept of dynamic tables and consideres how all
source/sink related classes play together.
We look forward to your feedback.
Regards,
Timo
