Re:FlinkSQL, define parallelism as HINT or CREATE

2024-12-26 Thread Xuyang
Hi, Currently, hints can modify the table options in the WITH parameters in a query. However, unfortunately, the Kafka source does not yet support related table options to specify the source's parallelism (see https://issues.apache.org/jira/browse/FLINK-33265 and https://github.com/apache/fl

Re:Can a Table Created in Java Code be Queried Using SQL Client?

2024-12-26 Thread Xuyang
Hi. In Java/Scala code, using `registerTable` and then querying in the SQL client will not work, as registerTable only creates a temporal table. To fulfill your requirements, you may need to use a persistent catalog[1] (such as a filesystem catalog) where you can create tables in your Java/Sc

Re:Re: Re: Flink and default parallelims, problems with state functions

2024-12-26 Thread Xuyang
You are right... Currently, the Flink SQL Client submits DML statements asynchronously to the Flink cluster, which means it is not possible to determine the final success or failure of the job directly from the console. -- Best! Xuyang 在 2024-12-26 20:38:56,"Guillermo Ortiz Fer

Can a Table Created in Java Code be Queried Using SQL Client?

2024-12-26 Thread Vinay Agarwal
Hello, I am looking for an elaboration, or work around, of an old answer to this question here ( https://stackoverflow.com/questions/56049472/create-sql-table-from-a-datastream-in-a-java-scala-program-and-query-it-from-sql) which mentions that registering the table can make it visible to the SQL c

Re: Re: Flink and default parallelims, problems with state functions

2024-12-26 Thread Guillermo Ortiz Fernández
it was my mistake, number and taskslot and parallelism wasn't right and I was inserting data from FlinkSQL. The insert didn't have enough resources and failed but from flinksql console wasn't reported any error. El mar, 24 dic 2024 a las 3:03, Xuyang () escribió: > Hi, > > Did you observe that th

FlinkSQL, define parallelism as HINT or CREATE

2024-12-26 Thread Guillermo Ortiz Fernández
I'm using Flink SQL and have several Kafka topics with different partition counts, ranging from 4 to 320 partitions. Is it possible to specify the level of parallelism as a HINT in queries or CREATE statements? If I don't define any, it defaults to parallelism.default. However, since the entire pro

Jobmanager & taskmanager restarts

2024-12-26 Thread banu priya
Hi All, I have a jobmanager and 3 taskmanager pods running in Kubernetes. Suddenly my taskmanager has following logs and it got restarted. ``` 2024-12-25 23:11:27.836 WARN org.apache.pekko.remote.transport.netty.NettyTransport [] - Remote connection to [/10.68.16.22:55200] failed with org