Thanks for you explain.
It seems that what you actually need is a general connector parameter in 'WITH'
block like 'filter-pushdown-enabled', and the specific connector rejects all
filters that could otherwise be pushed down in the connector.
Please forgive me if I'm wrong.
--
Best！
Xuyang
在 2023-10-24 14:34:15，"Jiabao Sun" <[email protected]> 写道：
>Thanks Xuyang,
>
>The table.optimizer.source.predicate-pushdown-enabled options do not provide
>fine-grained configuration for each source.
>
>Suppose we have an SQL query with two sources: Kafka and a database (CDC).
>The database is sensitive to pressure, and we want to configure it to not
>perform filter pushdown to the database source.
>However, we still want to perform filter pushdown to the Kafka source to
>decrease network IO.
>
>
>Best,
>Jiabao
> <https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/dev/table/config/#table-optimizer-source-predicate-pushdown-enabled>
>
>> 2023年10月24日 14:24，Xuyang <[email protected]> 写道：
>>
>> Hi, the existant configuration
>> 'table.optimizer.source.predicate-pushdown-enabled' seems to do what you
>> want.
>> Can you describe more clearly the difference between what you want and this
>> configuration ?
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> Best！
>> Xuyang
>>
>>
>>
>>
>>
>> At 2023-10-24 14:12:14, "Jiabao Sun" <[email protected]> wrote:
>>> Hi Devs,
>>>
>>> I would like to start a discussion on support configuration to disable
>>> filter pushdown for Table/SQL Sources[1].
>>>
>>> Currently, Flink SQL does not support the ability for users to enable or
>>> disable filter pushdown.
>>> However, filter pushdown has some side effects, such as additional
>>> computational pressure on external systems.
>>> Moreover, Improper queries can lead to issues such as full table scans,
>>> which in turn can impact the stability of external systems.
>>>
>>> I propose to support configuration to disable filter push down for
>>> Table/SQL sources to let user decide whether to perform filter pushdown.
>>>
>>> Looking forward to your feedback.
>>>
>>> [1]
>>> https://docs.google.com/document/d/1QsbOi9InvmfwFr8YbrnnXOKLPnb8JnqhXIMbGd68SFU/edit?usp=sharing
>>>
>>> Best,
>>> Jiabao
>
