Re: [DISCUSS] FLIP-515: Dynamic Kafka Sink

Your explanation is helpful. For question3, I think the key lies in whether the situation where MDS is unavailable is considered permissible, as subsequent updates to DynamicKafkaWriteState depend on this component. I approve of the usage of state if it is acceptable (for fault tolerance reasons).

[VOTE] FLIP-515: Dynamic Kafka Sink

Hi devs, I would like to start the vote for FLIP-515: Dynamic Kafka Sink [1] This FLIP was discussed in this thread [2]. The vote will be open for at least 72 hours unless there is an objection or insufficient votes. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-515:+Dynamic+Kafka+

[ANNOUNCE] New Apache Flink PMC Member - Sergey Nuyanzin

On behalf of the PMC, I'm very happy to announce a new Apache Flink PMC Member - Sergey Nuyanzin. Sergey has been a dedicated and invaluable contributor to the Flink community for several years. He became a committer in 2023 and has been one of the most active contributors, particularly in the are

[ANNOUNCE] New Apache Flink PMC Member - Zakelly Lan

On behalf of the PMC, I'm very happy to announce a new Apache Flink PMC Member - Zakelly Lan. Zakelly has been a dedicated and invaluable contributor to the Flink community for over five years. He became a committer early 2024 and is one of our most active contributors in the last year. Zakelly's

Re: [DISCUSS] FLIP-514: Custom Evaluator plugin for Flink Autoscaler

Hi Max, Good morning! Thanks so much for your review and feedback! I agree on proposal being generic and might not benefit a normal user without having a decent understanding of how autoscaler’s logic works. I had actually planned to contribute some predictive implementations using the custom

Re: Re: [VOTE] FLIP-512: Add meta information to SQL state connector

+1 (binding) Best, Ferenc On Monday, March 31st, 2025 at 13:06, Rui Fan <1996fan...@gmail.com> wrote: > > > +1(binding) > > Best, > Rui > > On Mon, Mar 31, 2025 at 5:42 PM Andrei Kaigorodov < > andreykaygoro...@gmail.com> wrote: > > > +1 (non-binding) > > > > Best regards, > > > > Andr

2025/03/27 CHI Eastern time zone workgroup meeting minutes

Hi, The 17th 30 minute working meeting of the Community Health Initiative (CHI) for Eastern time zone. * Thomas Cooper merged PR to change the stale bot numbers to the Apache standard * David Radley

Re: [VOTE] FLIP-432: Faster Checkpoint & Recovery for Disaggregated State

+1 (binding) Best, Zakelly On Thu, Apr 3, 2025 at 1:33 PM Yuan Mei wrote: > Thanks for driving this :-) > > +1 > > > > On Thu, Apr 3, 2025 at 12:34 PM Han Yin wrote: > > > Hi everyone, > > > > I'd like to start a vote on the FLIP-432: Faster Checkpoint & Recovery > for > > Disaggregated State

[jira] [Created] (FLINK-37534) Add Liniting checks to all PRs

Thomas Cooper created FLINK-37534: - Summary: Add Liniting checks to all PRs Key: FLINK-37534 URL: https://issues.apache.org/jira/browse/FLINK-37534 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-37607) Blocklist timeout check may lost

Heart Zhou created FLINK-37607: -- Summary: Blocklist timeout check may lost Key: FLINK-37607 URL: https://issues.apache.org/jira/browse/FLINK-37607 Project: Flink Issue Type: Bug Compon

Re: [DISCUSS] FLIP-511: Support transaction id pooling in Kafka connector

Hi David, Thank you for taking your time to go over the proposal. > * Can we check that we have the appropriate permissions and the > appropriate Kafka level(at least API key 66) to validate this option. That's probably possible but we can do that only during runtime (we may not have access

[jira] [Created] (FLINK-37557) ResolvedSchema#getPrimaryKeyIndexes does not filter for physical columns

Dawid Wysakowicz created FLINK-37557: Summary: ResolvedSchema#getPrimaryKeyIndexes does not filter for physical columns Key: FLINK-37557 URL: https://issues.apache.org/jira/browse/FLINK-37557 Proj

Executing PyFlink Cluster

Hello, I've been running into a wall for the past few days trying to get my PyFlink cluster running in a Docker container on my local machine. However, whenever I get to the point in my code where I need to run env.execute(), I see the following error (full stack trace at bottom of email): ```

Re: [DISCUSS] FLIP-515: Dynamic Kafka Sink

Hi, Matyas. Thanks for driving this. I have some questions and I hope you can help explain them: 1. The dynamic Kafka Sink looks like only be used in DataStream jobs currently, will it be considered to be supported in TableAPI in the future? 2. Even though we have Kafka MetaData Service, due to th

[jira] [Created] (FLINK-37586) YAML pipeline supports passing extra options to UDFs

yux created FLINK-37586: --- Summary: YAML pipeline supports passing extra options to UDFs Key: FLINK-37586 URL: https://issues.apache.org/jira/browse/FLINK-37586 Project: Flink Issue Type: Improvement

[VOTE] Release flink-connector-elasticsearch v4.0.0, release candidate #1

Hi everyone, Please review and vote on the release candidate #1 for v4.0.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) This release is mainly for flink 2.0. The complete staging area is available for your review, which inclu

[jira] [Created] (FLINK-37584) Bump assertj-core version from 3.23.1 to 3.27.3

Siddharth R created FLINK-37584: --- Summary: Bump assertj-core version from 3.23.1 to 3.27.3 Key: FLINK-37584 URL: https://issues.apache.org/jira/browse/FLINK-37584 Project: Flink Issue Type: Imp

Re: [VOTE] Release flink-connector-kudu v2.0.0, release candidate #1

+1 (binding) Verified: - Checkpoints, signatures - Checked notice files + no binaries in source release - Built from source - Verified release tag, release notes and website PR Cheers Gyula On Fri, Mar 28, 2025 at 2:34 PM Ferenc Csaky wrote: > Hi everyone, > > Please review and vote on rel

Re: [VOTE] FLIP-432: Faster Checkpoint & Recovery for Disaggregated State

+1 (binding) Best, Yanfei Zakelly Lan 于2025年4月3日周四 16:26写道： > > +1 (binding) > > > Best, > Zakelly > > On Thu, Apr 3, 2025 at 1:33 PM Yuan Mei wrote: > > > Thanks for driving this :-) > > > > +1 > > > > > > > > On Thu, Apr 3, 2025 at 12:34 PM Han Yin wrote: > > > > > Hi everyone, > > > > > > I

Re: [VOTE] FLIP-511: Support transaction id pooling in Kafka connector

Thank you everyone for voting. I will close this thread now and announce the results in a different thread. On Fri, Mar 28, 2025 at 6:11 PM Ahmed Hamdy wrote: > > +1 (non-binding) > > Thanks Arvid > Best Regards > Ahmed Hamdy > > > On Thu, 27 Mar 2025 at 16:11, Aleksandr Savonin wrote: > > > +1

[jira] [Created] (FLINK-37524) State Serializer in ForState should restore from oldStateInfo in Checkpoint

Yue Ma created FLINK-37524: -- Summary: State Serializer in ForState should restore from oldStateInfo in Checkpoint Key: FLINK-37524 URL: https://issues.apache.org/jira/browse/FLINK-37524 Project: Flink

Re: Re: [VOTE] FLIP-512: Add meta information to SQL state connector

+1(binding) Best, Shengkai Ferenc Csaky 于2025年3月31日周一 20:09写道： > +1 (binding) > > Best, > Ferenc > > > > > On Monday, March 31st, 2025 at 13:06, Rui Fan <1996fan...@gmail.com> > wrote: > > > > > > > +1(binding) > > > > Best, > > Rui > > > > On Mon, Mar 31, 2025 at 5:42 PM Andrei Kaigorodov < >

Re: [VOTE] FLIP-512: Add meta information to SQL state connector

+1 (binding) Gyula On Mon, Mar 31, 2025 at 8:03 AM Gabor Somogyi wrote: > Hi All, > > I'd like to start a vote on FLIP-512: Add meta information to SQL state > connector [1] > which has been discussed in this thread [2]. > > The vote will be open for at least 72 hours unless there is an objecti

Re: Process table function scan operation in batch mode is not supported

Hi Gabor, great that you already try out PTFs. I'm in the process of writing documentation for it. Including a list of limitations. Please note that PTF won't be support in batch mode in the first phase. For stateful PTFs we would need to use a batch state backend and also other code paths a

PR backlog

Hi, I notice the new numbers on the stalebot are having an effect on our backlog. We had over 1000 open non-draft PRs last week. We now have 603 PRs open and non-draft! We expect a couple of hundred more to close out in the near future. This came out of the Community Health Initiative (CHI) wor

[jira] [Created] (FLINK-37606) Blocklist timeout check may lost

Heart Zhou created FLINK-37606: -- Summary: Blocklist timeout check may lost Key: FLINK-37606 URL: https://issues.apache.org/jira/browse/FLINK-37606 Project: Flink Issue Type: Bug Compon

[jira] [Created] (FLINK-37538) Get Schame fail if tables.exclude contain them

linqigeng created FLINK-37538: - Summary: Get Schame fail if tables.exclude contain them Key: FLINK-37538 URL: https://issues.apache.org/jira/browse/FLINK-37538 Project: Flink Issue Type: Bug

Re: [EXTERNAL] Re: Executing PyFlink Cluster

Backported to 1.19 and 1.20 so we're fine here. @Ferenc, thanks for the backports👍 BR, G On Wed, Apr 2, 2025 at 8:08 PM Gabor Somogyi wrote: > Yeah, exactly. Go ahead and ping me, we'll punt it in... > > On Wed, Apr 2, 2025 at 7:17 PM Ferenc Csaky > wrote: > >> AFAIK the standard YAML config

Re: [DISCUSS] FLIP-515: Dynamic Kafka Sink

1. Yes, we can add Table API support in the future 2. The failover is coordinated by MDS itself 3. State is there to support situations where MDS is not accessible, it stores the cached metadata in the state I hope it clarifies, Matyas On Sat, Apr 5, 2025 at 9:55 AM Yanquan Lv wrote: > Hi, Maty

[jira] [Created] (FLINK-37618) Support PTFs INTERVAL argument

juntaozhang created FLINK-37618: --- Summary: Support PTFs INTERVAL argument Key: FLINK-37618 URL: https://issues.apache.org/jira/browse/FLINK-37618 Project: Flink Issue Type: Sub-task C

[jira] [Created] (FLINK-37589) Job submission via REST API is not thread safe

Ilya Soin created FLINK-37589: - Summary: Job submission via REST API is not thread safe Key: FLINK-37589 URL: https://issues.apache.org/jira/browse/FLINK-37589 Project: Flink Issue Type: Bug