Hi Timo, Thank you for the proposal. I think it is an important improvement that will benefit many parts of the Table API. The proposal looks really good to me and personally I would be comfortable with voting on the current state.
Best, Dawid On 23/03/2020 18:53, Timo Walther wrote: > Hi everyone, > > I received some questions around how the new interfaces play together > with formats and their factories. > > Furthermore, for MySQL or Postgres CDC logs, the format should be able > to return a `ChangelogMode`. > > Also, I incorporated the feedback around the factory design in general. > > I added a new section `Factory Interfaces` to the design document. > This should be helpful to understand the big picture and connecting > the concepts. > > Please let me know what you think? > > Thanks, > Timo > > > On 18.03.20 13:43, Timo Walther wrote: >> Hi Benchao, >> >> this is a very good question. I will update the FLIP about this. >> >> The legacy planner will not support the new interfaces. It will only >> support the old interfaces. With the next release, I think the Blink >> planner is stable enough to be the default one as well. >> >> Regards, >> Timo >> >> On 18.03.20 08:45, Benchao Li wrote: >>> Hi Timo, >>> >>> Thank you and others for the efforts to prepare this FLIP. >>> >>> The FLIP LGTM generally. >>> >>> +1 for moving blink data structures to table-common, it's useful to >>> udf too >>> in the future. >>> A little question is, do we plan to support the new interfaces and data >>> types in legacy planner? >>> Or we only plan to support these new interfaces in blink planner. >>> >>> And using primary keys from DDL instead of derived key information from >>> each query is also a good idea, >>> we met some use cases where this does not works very well before. >>> >>> This FLIP also makes the dependencies of table modules more clear, I >>> like >>> it very much. >>> >>> Timo Walther <twal...@apache.org> 于2020年3月17日周二 上午1:36写道： >>> >>>> Hi everyone, >>>> >>>> I'm happy to present the results of long discussions that we had >>>> internally. Jark, Dawid, Aljoscha, Kurt, Jingsong, me, and many more >>>> have contributed to this design document. >>>> >>>> We would like to propose new long-term table source and table sink >>>> interfaces: >>>> >>>> >>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-95%3A+New+TableSource+and+TableSink+interfaces >>>> >>>> >>>> This is a requirement for FLIP-105 and finalizing FLIP-32. >>>> >>>> The goals of this FLIP are: >>>> >>>> - Simplify the current interface architecture: >>>> - Merge upsert, retract, and append sinks. >>>> - Unify batch and streaming sources. >>>> - Unify batch and streaming sinks. >>>> >>>> - Allow sources to produce a changelog: >>>> - UpsertTableSources have been requested a lot by users. Now >>>> is the >>>> time to open the internal planner capabilities via the new interfaces. >>>> - According to FLIP-105, we would like to support changelogs for >>>> processing formats such as Debezium. >>>> >>>> - Don't rely on DataStream API for source and sinks: >>>> - According to FLIP-32, the Table API and SQL should be >>>> independent >>>> of the DataStream API which is why the `table-common` module has no >>>> dependencies on `flink-streaming-java`. >>>> - Source and sink implementations should only depend on the >>>> `table-common` module after FLIP-27. >>>> - Until FLIP-27 is ready, we still put most of the interfaces in >>>> `table-common` and strictly separate interfaces that communicate >>>> with a >>>> planner and actual runtime reader/writers. >>>> >>>> - Implement efficient sources and sinks without planner dependencies: >>>> - Make Blink's internal data structures available to connectors. >>>> - Introduce stable interfaces for data structures that can be >>>> marked as `@PublicEvolving`. >>>> - Only require dependencies on `flink-table-common` in the >>>> future >>>> >>>> It finalizes the concept of dynamic tables and consideres how all >>>> source/sink related classes play together. >>>> >>>> We look forward to your feedback. >>>> >>>> Regards, >>>> Timo >>>> >>> >>> >
signature.asc
Description: OpenPGP digital signature
