Thanks for bringing this up. this is a good feature. but i have two questions: 1. if the two insert into with same columns, the result is not nondeterminism. will it check in planner and throw exception 2. some sink connectors can not supports it like queue such as kafka compacted topic. will also it check in planner and throw exception
Lincoln Lee <lincoln.8...@gmail.com> 于2023年3月7日周二 14:53写道： > Hi Aitozi, > > Thanks for your feedback! Yes, including HBase and JDBC connector, they > can be considered for support in the next step (JDBC as as a standard > protocol supported not only in traditional databases, but also in more and > more new types of storage). Considering the ongoing externalizing of > connectors and the release cycles of the connectors are decoupled with the > release cycle of Flink, we can initiate corresponding support issues for > specific connectors to follow up on support after finalizing the API > changes, WDYT? > > Best, > Lincoln Lee > > > Hang Ruan <ruanhang1...@gmail.com> 于2023年3月7日周二 12:14写道： > > > Hi, Lincoln, > > > > Thanks for bringing this up. It looks good to me. I also agree with > > Jingsong's suggestion. > > > > Best, > > Hang > > > > Jingsong Li <jingsongl...@gmail.com> 于2023年3月7日周二 11:15写道： > > > > > Wow, we have 300 FLIPs... > > > > > > Thanks Lincoln, > > > > > > Have you considered returning an Optional<int[][]>? > > > > > > Empty array looks a little weird to me. > > > > > > Best, > > > Jingsong > > > > > > On Tue, Mar 7, 2023 at 10:32 AM Aitozi <gjying1...@gmail.com> wrote: > > > > > > > > Hi Lincoln, > > > > Thank you for sharing this FLIP. Overall, it looks good to me. I > > have > > > > one question: with the introduction of this interface, > > > > will any existing Flink connectors need to be updated in order to > take > > > > advantage of its capabilities? For example, HBase. > > > > > > > > yuxia <luoyu...@alumni.sjtu.edu.cn> 于2023年3月7日周二 10:01写道： > > > > > > > > > Thanks. It makes sense to me. > > > > > > > > > > Best regards, > > > > > Yuxia > > > > > > > > > > ----- 原始邮件 ----- > > > > > 发件人: "Lincoln Lee" <lincoln.8...@gmail.com> > > > > > 收件人: "dev" <dev@flink.apache.org> > > > > > 发送时间: 星期一, 2023年 3 月 06日 下午 10:26:26 > > > > > 主题: Re: [DISCUSS] FLIP-300: Add targetColumns to > > > DynamicTableSink#Context > > > > > to solve the null overwrite problem of partial-insert > > > > > > > > > > hi yuxia, > > > > > > > > > > Thanks for your feedback and tracking the issue of update > statement! > > > I've > > > > > updated the FLIP[1] and also the poc[2]. > > > > > Since the bug and flip are orthogonal, we can focus on finalizing > the > > > api > > > > > changes first, and then work on the flip implementation and bugfix > > > > > separately, WDYT? > > > > > > > > > > [1] > > > > > > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240885081 > > > > > [2] https://github.com/apache/flink/pull/22041 > > > > > > > > > > Best, > > > > > Lincoln Lee > > > > > > > > > > > > > > > yuxia <luoyu...@alumni.sjtu.edu.cn> 于2023年3月6日周一 21:21写道： > > > > > > > > > > > Hi, Lincoln. > > > > > > Thanks for bringing this up. +1 for this FLIP, it's helpful for > > > external > > > > > > storage system to implement partial update. > > > > > > The FLIP looks good to me. I only want to add one comment, update > > > > > > statement also doesn't support updating nested column, I have > > created > > > > > > FLINK-31344[1] to track it. > > > > > > Maybe we also need to explain it in this FLIP. > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-31344 > > > > > > > > > > > > Best regards, > > > > > > Yuxia > > > > > > > > > > > > ----- 原始邮件 ----- > > > > > > 发件人: "Lincoln Lee" <lincoln.8...@gmail.com> > > > > > > 收件人: "dev" <dev@flink.apache.org> > > > > > > 发送时间: 星期五, 2023年 3 月 03日 下午 12:22:19 > > > > > > 主题: [DISCUSS] FLIP-300: Add targetColumns to > > > DynamicTableSink#Context to > > > > > > solve the null overwrite problem of partial-insert > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > This FLIP[1] aims to support connectors in avoiding overwriting > > > > > non-target > > > > > > columns with null values when processing partial column updates, > we > > > > > propose > > > > > > adding information on the target column list to > > > DynamicTableSink#Context. > > > > > > > > > > > > FLINK-18726[2] supports inserting statements with specified > column > > > list, > > > > > it > > > > > > fills null values (or potentially declared default values in the > > > future) > > > > > > for columns not appearing in the column list of insert statement > to > > > the > > > > > > target table. > > > > > > But this behavior does not satisfy some partial column update > > > > > requirements > > > > > > of some storage systems which allow storing null values. The > > problem > > > is > > > > > > that connectors cannot distinguish whether the null value of a > > > column is > > > > > > really from the user's data or whether it is a null value > populated > > > > > because > > > > > > of partial insert behavior. > > > > > > > > > > > > Looking forward to your comments or feedback. > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240885081 > > > > > > [2] https://issues.apache.org/jira/browse/FLINK-18726 > > > > > > > > > > > > Best, > > > > > > Lincoln Lee > > > > > > > > > > > > > > > > >
