Thanks Xuyang, The table.optimizer.source.predicate-pushdown-enabled options do not provide fine-grained configuration for each source.
Suppose we have an SQL query with two sources: Kafka and a database (CDC). The database is sensitive to pressure, and we want to configure it to not perform filter pushdown to the database source. However, we still want to perform filter pushdown to the Kafka source to decrease network IO. Best, Jiabao <https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/dev/table/config/#table-optimizer-source-predicate-pushdown-enabled> > 2023年10月24日 14:24，Xuyang <xyzhong...@163.com> 写道： > > Hi, the existant configuration > 'table.optimizer.source.predicate-pushdown-enabled' seems to do what you > want. > Can you describe more clearly the difference between what you want and this > configuration ? > > > > > > > > -- > > Best！ > Xuyang > > > > > > At 2023-10-24 14:12:14, "Jiabao Sun" <jiabao....@xtransfer.cn.INVALID> wrote: >> Hi Devs, >> >> I would like to start a discussion on support configuration to disable >> filter pushdown for Table/SQL Sources[1]. >> >> Currently, Flink SQL does not support the ability for users to enable or >> disable filter pushdown. >> However, filter pushdown has some side effects, such as additional >> computational pressure on external systems. >> Moreover, Improper queries can lead to issues such as full table scans, >> which in turn can impact the stability of external systems. >> >> I propose to support configuration to disable filter push down for Table/SQL >> sources to let user decide whether to perform filter pushdown. >> >> Looking forward to your feedback. >> >> [1] >> https://docs.google.com/document/d/1QsbOi9InvmfwFr8YbrnnXOKLPnb8JnqhXIMbGd68SFU/edit?usp=sharing >> >> Best, >> Jiabao
