Re: [DISCUSSION] Upgrade to Kryo 5 for Flink 2.0

2025-02-25 Thread Nick Nezis
Thanks Martijn. That's really great context. In that case, then I'll change my previous opinion. I agree that we should proceed with the simpler pull request and get it into the Flink 2.0 release. On 2025/02/25 14:06:20 Martijn Visser wrote: > Hi all, > > For the record, I don't think we have

Kubernetes operator questions

2025-02-25 Thread santhosh venkat
Hi, I am a newbie to the Flink-kubernetes operator. We are planning to adopt/use it in my company, and it would be great if someone can help answer my questions. 1. It seems like the kubernetes operator is coupled with the auto-scaler. The operator is managing the lifecycle of the Flink job

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.11.0, release candidate #1

2025-02-25 Thread Tom Cooper
+1 (non-binding) - Verified the sha512's for the Helm archive and source archive. - Verified that source archive builds correctly with maven 3.9.9 and Java 17.0.2. - Installed the Helm chart in minikube and deployed several example FlinkDeployments from the examples directory (using Flink 1_20).

Re: FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-02-25 Thread Sergey Nuyanzin
+1 for such improvement On Mon, Feb 24, 2025 at 12:01 PM Dawid Wysakowicz wrote: > > Hi everyone, > > I would like to initiate a discussion for the FLIP-510[1] below, which aims > on optimising certain use cases in SQL which at the moment add > ChangelogNormalize, but don't necessarily need to do

Re: [DISCUSSION] Upgrade to Kryo 5 for Flink 2.0

2025-02-25 Thread Martijn Visser
Hi all, For the record, I don't think we have a guarantee around backwards compatibility for Flink 2.0 anyway, given that we upgraded Scala to the latest version (because of the bump to JDK 17) and that will potentially break savepoints when using Scala. So I think we should also put this in for F

[jira] [Created] (FLINK-37385) Add support for smile format in flink main repo

2025-02-25 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-37385: --- Summary: Add support for smile format in flink main repo Key: FLINK-37385 URL: https://issues.apache.org/jira/browse/FLINK-37385 Project: Flink Issue T

[jira] [Created] (FLINK-37384) the execution time of DefaultVertexParallelismAndInputInfosDecider exceeds expectations in specific scenarios

2025-02-25 Thread Lei Yang (Jira)
Lei Yang created FLINK-37384: Summary: the execution time of DefaultVertexParallelismAndInputInfosDecider exceeds expectations in specific scenarios Key: FLINK-37384 URL: https://issues.apache.org/jira/browse/FLINK-3

[jira] [Created] (FLINK-37383) ThrottleIterator from examples/utils does not properly throttle on next window after throttle is applied

2025-02-25 Thread Rafael Zimmermann (Jira)
Rafael Zimmermann created FLINK-37383: - Summary: ThrottleIterator from examples/utils does not properly throttle on next window after throttle is applied Key: FLINK-37383 URL: https://issues.apache.org/jira/br

Re: [DISCUSSION] Upgrade to Kryo 5 for Flink 2.0

2025-02-25 Thread Zhanghao Chen
Hi Gyula, Thanks for bringing this up! Definitely +1 for upgrading Kryo in Flink 2.0. As a side note, it might be useful to introduce customizable generic serializer support like Spark, where you can switch to your own serializer via the "spark.serializer" [1] option. Users starting new applica

[jira] [Created] (FLINK-37382) Early optimization for adaptive join when one input is below broadcast threshold

2025-02-25 Thread xingbe (Jira)
xingbe created FLINK-37382: -- Summary: Early optimization for adaptive join when one input is below broadcast threshold Key: FLINK-37382 URL: https://issues.apache.org/jira/browse/FLINK-37382 Project: Flink

Re: [DISCUSS] Pluggable Batching for Async Sink in Flink

2025-02-25 Thread Poorvank Bhatia
Hey Folks, I've made the necessary updates to the proposal document. Please review and share your thoughts. @Arvid, regarding the discussion on SPI and Reflection—one key limitation is that both approaches require BatchCreator (or any other pluggable component) to either have a *no-argument const

Re: [DISCUSS] Pluggable Batching for Async Sink in Flink

2025-02-25 Thread Arvid Heise
Hi Poorvank, I don't have strong feelings about the actual choice of technology. SPI is used rather heavily in Flink Table/SQL and is also used in similar projects more and more. But SPI also uses reflection under the hood, so it doesn't make a huge difference except from a usability perspective.

[VOTE] Apache Flink Kubernetes Operator Release 1.11.0, release candidate #1

2025-02-25 Thread Gyula Fóra
Hi Everyone, Please review and vote on the release candidate #1 for the version 1.11.0 of Apache Flink Kubernetes Operator, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) **Release Overview** As an overview, the release consists of t