Thanks Leonard, I have added sub-sections under the “Motivation" chapter to describe the differences about them.
Best, Jiabao > 2023年10月24日 16:38，Leonard Xu <xbjt...@gmail.com> 写道： > > Thanks @Jiabao for kicking off this discussion. > > Could you add a section to explain the difference between proposed connector > level config `scan.filter-push-down.enabled` and existing query level config > `table.optimizer.source.predicate-pushdown-enabled` ? > > Best, > Leonard > >> 2023年10月24日 下午4:18，Jiabao Sun <jiabao....@xtransfer.cn.INVALID> 写道： >> >> Hi Devs, >> >> I would like to start a discussion on FLIP-377: support configuration to >> disable filter pushdown for Table/SQL Sources[1]. >> >> Currently, Flink Table/SQL does not expose fine-grained control for users to >> enable or disable filter pushdown. >> However, filter pushdown has some side effects, such as additional >> computational pressure on external systems. >> Moreover, Improper queries can lead to issues such as full table scans, >> which in turn can impact the stability of external systems. >> >> Suppose we have an SQL query with two sources: Kafka and a database. >> The database is sensitive to pressure, and we want to configure it to not >> perform filter pushdown to the database source. >> However, we still want to perform filter pushdown to the Kafka source to >> decrease network IO. >> >> I propose to support configuration to disable filter push down for Table/SQL >> sources to let user decide whether to perform filter pushdown. >> >> Looking forward to your feedback. >> >> [1] >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=276105768 >> >> Best, >> Jiabao >
