I'm going to follow the guidelines you mentioned. On the other hand, do you
know if there is any real performance comparison between Flink SQL and
Flink? By using 100% Flink SQL, I think we gain a lot of ease of use, but
we might be losing performance. Additionally, there are features available
in
I have been configuring the Flink cluster (application mode) to process the
Kafka data volume. The current configuration consists of 16 pods, each
with *12GB
of memory and 2 CPUs*. Each TaskManager has *4 slots*.
All processing is done using *Flink SQL*, and since Kafka topics may
contain out-of-o
> The average output rate needed to avoid lag after filtering messages
should be around 60K messages per second. I’ve been testing different
configurations of parallelism, slots and pods (everything runs on
Kubernetes), but I’m far from achieving those numbers.
How are you partitioning your query?
After last checking it uses about 200-400 millicores each pod and 2.2Gb.
El mié, 29 ene 2025 a las 21:41, Guillermo Ortiz Fernández (<
guillermo.ortiz.f...@gmail.com>) escribió:
> I have a job entirely written in Flink SQL. The first part of the program
> processes 10 input topics and generates o
