Hi, 

Currently, hints can modify the  table options in the WITH parameters in a 
query. However, unfortunately, the Kafka source does not yet support related 
table options to specify the source's parallelism (see 
https://issues.apache.org/jira/browse/FLINK-33265 and 
https://github.com/apache/flink-connector-kafka/pull/134 ). You might consider 
patching this PR to the Kafka connector yourself, rebuilding your Kafka 
connector and using dynamic table option hints[1] to achieve your requirements.

[1] 
https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/hints/#dynamic-table-options




--

    Best!
    Xuyang




At 2024-12-26 20:24:56, "Guillermo Ortiz Fernández" 
<guillermo.ortiz.f...@gmail.com> wrote:

I'm using Flink SQL and have several Kafka topics with different partition 
counts, ranging from 4 to 320 partitions. Is it possible to specify the level 
of parallelism as a HINT in queries or CREATE statements? If I don't define 
any, it defaults to parallelism.default. However, since the entire program is 
defined in a .sql file, I want to set different levels for each query.



I understand that the parallelism level of a Kafka source matches the number of 
partitions in the topic, although I haven’t found this explicitly mentioned in 
the documentation.

Reply via email to