And which is the parallelism that flink is going to choose to Kafka topic right now?
El vie, 27 dic 2024, 7:47, Xuyang <xyzhong...@163.com> escribió: > Hi, > > Currently, hints can modify the table options in the WITH parameters in a > query. However, unfortunately, the Kafka source does not yet support > related table options to specify the source's parallelism (see > https://issues.apache.org/jira/browse/FLINK-33265 and > https://github.com/apache/flink-connector-kafka/pull/134 ). You might > consider patching this PR to the Kafka connector yourself, rebuilding your > Kafka connector and using dynamic table option hints[1] to achieve your > requirements. > > [1] > https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/hints/#dynamic-table-options > > > -- > Best！ > Xuyang > > > At 2024-12-26 20:24:56, "Guillermo Ortiz Fernández" < > guillermo.ortiz.f...@gmail.com> wrote: > > I'm using Flink SQL and have several Kafka topics with different partition > counts, ranging from 4 to 320 partitions. Is it possible to specify the > level of parallelism as a HINT in queries or CREATE statements? If I don't > define any, it defaults to parallelism.default. However, since the entire > program is defined in a .sql file, I want to set different levels for > each query. > > I understand that the parallelism level of a Kafka source matches the > number of partitions in the topic, although I haven’t found this explicitly > mentioned in the documentation. > >
