RE: TCP Socket stream scalability

Hello Community, Please share views for below mail. Rgds, Kamal From: Kamal Mittal via user Sent: 14 July 2023 12:55 PM To: user@flink.apache.org Subject: TCP Socket stream scalability Hello, TCP Socket stream can be scaled across task managers similarly to file enumerator and source reader

Re: Flink Logs are missing While writing to file - Large amount of Events

Hello Weihua, I tried to increase akka.framesize from default to 30MB but I still see few logs are missing from down stream operators if there's large amount of data processing. Regards,Madan  On Friday, 14 July 2023 at 11:45:36 am GMT-7, Madan D via user wrote: Hello Weihua, I see

Re: Using pre-registered schemas with avro-confluent-registry format is not possible

There’s no need to build Flink. Implement your own org.apache.flink.formats.avro.SchemaCoder, and the SchemaCoder.SchemaCoderProvider accepted by the constructor of org.apache.flink.formats.avro.RegistryAvroSerializationSchema. This way, you can replace the dangerous “register()” with simply “g

Re: Flink Logs are missing While writing to file - Large amount of Events

Hello Weihua, I see all events are written, which are processed right after the event has been consumed (source), but they are missing those that are coming from downstream operators. Regards,Madan   On Friday, 14 July 2023 at 02:16:11 am GMT-7, Weihua Hu wrote: Hi Madan Flink UI re

Need help to debug the issue 413 Request Entity Too Large

Hi Flink experts, I have been facing this issue for quite some days as I am not able to submit/run SQL Queries using REST CLIENT OR FLINK SQL CLIENT via SQL gateway. Here's my Setup Cluster Set up via k8s operator to deploy Session Cluster, Service gateway and ingress files. I am able to run ``

Issue with flink 1.16 and hive dialect

Hi All, I am not sure if this was already discussed in this forum. In our set up with 1.16.0 flink we have ensured that the setup has all the necessary things for Hive catalog to work. The flink dialect works fine functionally (with some issues will come to that later). But when i follow the ste

Re: Flink Logs are missing While writing to file - Large amount of Events

Hi Madan Flink UI reads the log file and displays it in the UI, so if the UI is accurate, There shouldn't be any issues with Flink writing files. You can check if there are any issues while transferring log file data to Splunk. Best, Weihua On Wed, Jul 12, 2023 at 1:02 AM Madan D via user wr

TCP Socket stream scalability

Hello, TCP Socket stream can be scaled across task managers similarly to file enumerator and source reader below? Job is submitted with TCP socket source function and a socket will bind on a port once and by a task manager. Is it possible to open socket at job manager and then scale / divide t