Re: [DISCUSS] FLIP-XXX Add K8S conditions to Flink CRD

Thank you so much, Ferenc , and regarding your observation , FLIP covers the changes in FlinkDeployment CR which defines Flink Application and Session cluster deployments, hence it is referred to as FlinkDeployment. Thank you Regards Lajith On Thu, Nov 7, 2024 at 12:46 AM Ferenc Csaky wrote: >

[jira] [Created] (FLINK-36670) Optimize port range out-of-bounds exception log

Gianzie created FLINK-36670: --- Summary: Optimize port range out-of-bounds exception log Key: FLINK-36670 URL: https://issues.apache.org/jira/browse/FLINK-36670 Project: Flink Issue Type: Improvement

Re: Flink technical debt.

Hello David, I believe this is a valuable initiative that will significantly enhance the codebase as well. I would also like to join this group and contribute to the community. Regards, Anu K T On Mon, Nov 4, 2024 at 8:36 PM David Radley wrote: > Hello, > I have been looking at the Flink Jira

Re: Flink technical debt.

Hi David, Proposal looks useful and valuable. I would like to join the group and contribute, It would be great to have a meeting to outline the goals and what group members can do to meet the goals. Thank you Regards Lajith K On Thu, Nov 7, 2024 at 1:50 AM Shalini wrote: > Hi David, > > I agree

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations Junrui~ Best regards, Wenjin > 2024年11月5日 19:59，Zhu Zhu 写道： > > Hi everyone, > > On behalf of the PMC, I'm happy to announce that Junrui Li has become a > new Flink Committer! > > Junrui has been an active contributor to the Apache Flink project for two > years. He had been th

[jira] [Created] (FLINK-36669) The flink version of ES should use parent's pom

Gianzie created FLINK-36669: --- Summary: The flink version of ES should use parent's pom Key: FLINK-36669 URL: https://issues.apache.org/jira/browse/FLINK-36669 Project: Flink Issue Type: Improvement

Re: Re: Re: [DISCUSS] FLIP-480: Support to deploy script in application mode

Hi, eveyone. The discussion doesn't receive any response for a while. I will close the discussion and start the vote tomorrow if the discussion doesn't receive any response today. Thanks all yours response! Best, Shengkai Shengkai Fang 于2024年11月5日周二 14:00写道： > Hi, Lincoln. Thanks for your resp

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations Junrui~ Best regards, Weijie Leonard Xu 于2024年11月6日周三 21:19写道： > Congratulations Junrui ! > > > Best, > Leonard > > > 2024年11月6日 下午8:52，Hong Liang 写道： > > > > Congratulations Junrui! > > > > Hong > > > > On Wed, Nov 6, 2024 at 12:36 PM Yanquan Lv wrote: > > > >> Congratulati

Re: Flink technical debt.

Hi David, I agree that this is a very valuable initiative, and I would like to join this working group as an opportunity to contribute to the community. Thanks, Shalini M On Thu, Nov 7, 2024 at 12:11 AM Ferenc Csaky wrote: > Hi, > > Thank you David for initiating this discussion. I think it wo

RE: Flink technical debt.

Hi David, Thank you for this initiative. You can count me in. I think that will help to prioritize and focus on what mater the most to the community. I did a quick search on Jira to help narrow down the list of Blocker and Critical issues based on the issue type: => 195 are not Closed, Done or

RE: [DISCUSS] FLIP-XXX Add K8S conditions to Flink CRD

Hi, I can help to create a FLIP page, from the gdoc, but one thing that I noteced is under "Session mode" both the text and the code snippets refer to "FlinkDeployment". I believe that should be "FlinkSessionJob". Best, Ferenc On Wednesday, November 6th, 2024 at 17:33, David Radley wrote: >

Re: Flink technical debt.

Hi, Thank you David for initiating this discussion. I think it would definitely worth some attention from the community to try to focus a bit more on these problems. I would be happy to participate in the calls, and help out with reviewing/merging PRs. Best, Ferenc On Wednesday, November 6t

Re: Flink technical debt.

Hi David, Tom, Regarding bots, I've seen that there is some JIRA bot which moves priority down over time. See https://issues.apache.org/jira/browse/FLINK-12173 as an example. I recently hit an issue where a new-to-Flink contributor had filed a ticket and put up a PR around six months ago and no

[jira] [Created] (FLINK-36668) Release flink-connector-aws v5.0.0

Hong Liang Teoh created FLINK-36668: --- Summary: Release flink-connector-aws v5.0.0 Key: FLINK-36668 URL: https://issues.apache.org/jira/browse/FLINK-36668 Project: Flink Issue Type: Technica

RE: Flink technical debt.

Hi Tom, Welcome 😊 Does anyone in the community know if we have considered running bots as Tom suggests, Kind regards, David. From: Tom Cooper Date: Wednesday, 6 November 2024 at 15:34 To: dev@flink.apache.org Subject: [EXTERNAL] Re: Flink technical debt. Hey David, As a newcomer to the

RE: Flink technical debt.

Hi Robert, Yes that sounds good. I agree we need to be visible on Jira and the mailing list – fully visible. Following up would mean that we chase the submitter? Another win would be if we can identify work that members of the group can do that would save time for the committers. This could inv

Re: [VOTE] Release flink-connector-prometheus v1.0.0, release candidate #1

Thanks for the thorough verification Danny! > - I note that the AWS SDK version differs between flink-connector- prometheus [1] and flink-connector-aws [2]. This is probably ok, but not ideal. We should find a strategy to align this in the future to avoid dependency issues. Thanks for the comment

[VOTE] Release flink-connector-aws v5.0.0, release candidate #3

Hi everyone, Please review and vote on release candidate #3 for flink-connector-aws v5.0.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1

RE: [DISCUSS] FLIP-XXX Add K8S conditions to Flink CRD

Hi lajith, Yes I like the simplicity of the current proposal. Hi Gyula, The next stage is to assign a Flip number and move the content of the google doc into the flip wiki. Unfortunately, as we are not committers, we are not authorized to do either of these activities. Are you able to copy this

Re: [DISCUSS] Changing return type of window properties (WINDOW_START/END)

Hi Dawid, thanks for starting this thread. It is very unfortunate that we did not spot this issue earlier when implementing window TVFs. But I agree that this is a bug. Unless somebody can provide a reason why window start/end always use TIMESTAMP and don't adapt to TIMESTAMP_LTZ? If I remem

[jira] [Created] (FLINK-36667) Remove `proto` schema definitions from source

Lorenzo Nicora created FLINK-36667: -- Summary: Remove `proto` schema definitions from source Key: FLINK-36667 URL: https://issues.apache.org/jira/browse/FLINK-36667 Project: Flink Issue Type:

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congrats, Junrui! Luke On Wed, Nov 6, 2024 at 8:20 PM Jing Ge wrote: > Congrats, Junrui! > > Best regards, > Jing > > On Wed, Nov 6, 2024 at 11:46 AM Yu Chen wrote: > > > Congratulations, Junrui! > > > > Best, > > Yu Chen > > > > > > > 2024年11月6日 16:32，Xuyang 写道： > > > > > > Congratulations,

[jira] [Created] (FLINK-36663) [Window]The first processWatermark() after Sliding Window restore may have extra expired data.

kaitian created FLINK-36663: --- Summary: [Window]The first processWatermark() after Sliding Window restore may have extra expired data. Key: FLINK-36663 URL: https://issues.apache.org/jira/browse/FLINK-36663

Re: Flink technical debt.

Hey David, As a newcomer to the community myself, the number of open PRs and JIRA issues is a bit daunting. So thanks for starting this initiative. I like the idea of tackling the mountain by triaging the PRs / JIRA Issues and I would be very happy to join the effort. However, I wonder if it w

Re: Flink technical debt.

Hey David, Thanks a lot for this initiative. What do you think about setting up a call weekly or every second week, open to the public, where we collaboratively review a backlog of Jira tickets and triage them into: - Closing, - following up, - assigning a committer owner? The purpose of the call

Re: Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations Junrui! Best Regards Ahmed Hamdy On Wed, 6 Nov 2024 at 08:32, Xuyang wrote: > Congratulations, Junrui! > > > -- > > Best！ > Xuyang > > > > > > 在 2024-11-06 14:34:36，"Yuepeng Pan" 写道： > >Congratulations! > > > > > >Best, > >Yuepeng Pan > > > > > > > >At 2024-11-06 14:08

Re: Kafka connector releases

Hi Yanquan, the current state of the 3.4.0 release is that it's still pending on the lineage PR [1] which I expect to be merged next week (the author is on vacation). The release cut would then happen right afterwards. After the release cut, we can then bump to 4.0.0-SNAPSHOT and Flink 2.0-previe

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations Junrui! Hong On Wed, Nov 6, 2024 at 12:36 PM Yanquan Lv wrote: > Congratulations, Junrui! > > Best, > Yanquan > > > 2024年11月5日 19:59，Zhu Zhu 写道： > > > > Hi everyone, > > > > On behalf of the PMC, I'm happy to announce that Junrui Li has become a > > new Flink Committer! > > > >

Re: [VOTE] Release flink-connector-aws v5.0.0, release candidate #2

Thank you for checking Danny. Good catch on the NOTICE files. I have created a JIRA to make this check automatic for all PRs [1] > Since we do not have state compatibility between the new and legacy source we need to make this clear in the docs. I do not see a task for this. I will continue to rev

[jira] [Created] (FLINK-36666) CI/CD should check NOTICE files for discrepancies.

Hong Liang Teoh created FLINK-3: --- Summary: CI/CD should check NOTICE files for discrepancies. Key: FLINK-3 URL: https://issues.apache.org/jira/browse/FLINK-3 Project: Flink Issu

Re: Kafka connector releases

Thanks @Yanquan for track this thread, and thanks @Arvid for the information, it makes sense to me Qingsheng will drive the release of flink-2.0-preview of Flink Kafka connector, and I’d like assist it too. Best, Leonard > 2024年11月6日 下午6:58，Arvid Heise 写道： > > Hi Yanquan, > > the current s

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Welcome Junrui! On Wed, Nov 6, 2024 at 1:53 PM Hong Liang wrote: > Congratulations Junrui! > > Hong > > On Wed, Nov 6, 2024 at 12:36 PM Yanquan Lv wrote: > > > Congratulations, Junrui! > > > > Best, > > Yanquan > > > > > 2024年11月5日 19:59，Zhu Zhu 写道： > > > > > > Hi everyone, > > > > > > On beha

[DISCUSS] Changing return type of window properties (WINDOW_START/END)

Hi all, I wanted to bring your attention to FLINK-36665[1]. I believe the current behaviour is confusing and I'd like to fix it. However, since window operations are a very important feature I'd like to gather feedback on to what extent we should keep backwards compatibility. 1. How should ne

[jira] [Created] (FLINK-36665) WINDOW_START, WINDOW_END always return TIMESTAMP(3)

Dawid Wysakowicz created FLINK-36665: Summary: WINDOW_START, WINDOW_END always return TIMESTAMP(3) Key: FLINK-36665 URL: https://issues.apache.org/jira/browse/FLINK-36665 Project: Flink I

Re: [VOTE] Release flink-connector-prometheus v1.0.0, release candidate #1

Thanks for driving this Hong (and Lorenzo), nice to see the new connector launching. +1 binding - I note that the AWS SDK version differs between flink-connector-prometheus [1] and flink-connector-aws [2]. This is probably ok, but not ideal. We should find a strategy to align this in the future t

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations, Junrui! Best, Yu Chen > 2024年11月6日 16:32，Xuyang 写道： > > Congratulations, Junrui! > > > -- > >Best！ >Xuyang > > > > > > 在 2024-11-06 14:34:36，"Yuepeng Pan" 写道： >> Congratulations! >> >> >> Best, >> Yuepeng Pan >> >> >> >> At 2024-11-06 14:08:41, "Xia Sun"

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations, Junrui! Best, Yanquan > 2024年11月5日 19:59，Zhu Zhu 写道： > > Hi everyone, > > On behalf of the PMC, I'm happy to announce that Junrui Li has become a > new Flink Committer! > > Junrui has been an active contributor to the Apache Flink project for two > years. He had been the driv

Re: [VOTE] Release flink-connector-aws v5.0.0, release candidate #2

Thanks Hong, > 1/ New connectors (SQS sink, DynamoDB Streams source). We are also adding more functionality to the KDS source based on FLIP-27 interfaces. These would not warrant a v5.x, but the breaking change to the DynamicTableSource (2) does. Since we do not have state compatibility between t

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congrats, Junrui! Best regards, Jing On Wed, Nov 6, 2024 at 11:46 AM Yu Chen wrote: > Congratulations, Junrui! > > Best, > Yu Chen > > > > 2024年11月6日 16:32，Xuyang 写道： > > > > Congratulations, Junrui! > > > > > > -- > > > >Best！ > >Xuyang > > > > > > > > > > > > 在 2024-11-06 14:34:36，"Y

[jira] [Created] (FLINK-36664) [Window]The window with window_offset will lose data.

kaitian created FLINK-36664: --- Summary: [Window]The window with window_offset will lose data. Key: FLINK-36664 URL: https://issues.apache.org/jira/browse/FLINK-36664 Project: Flink Issue Type: Bug

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations Junrui ! Best, Leonard > 2024年11月6日 下午8:52，Hong Liang 写道： > > Congratulations Junrui! > > Hong > > On Wed, Nov 6, 2024 at 12:36 PM Yanquan Lv wrote: > >> Congratulations, Junrui! >> >> Best, >> Yanquan >> >>> 2024年11月5日 19:59，Zhu Zhu 写道： >>> >>> Hi everyone, >>> >>> On

Re:Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

Congratulations, Junrui! -- Best！ Xuyang 在 2024-11-06 14:34:36，"Yuepeng Pan" 写道： >Congratulations! > > >Best, >Yuepeng Pan > > > >At 2024-11-06 14:08:41, "Xia Sun" wrote: >>Congratulations Junrui! >> >> >>Best, >>Xia >> >>Zakelly Lan 于2024年11月6日周三 09:57写道： >> >>> Congratulations,

Re: [DISCUSS] Is it a bug that the AdaptiveScheduler does not prioritize releasing TaskManagers during downscaling in Application mode?

Thanks Yuepeng for the PR and starting this discussion! And thanks Gyula and Yuanfeng for the input! I also agree to fix this behaviour in the 1.x line. The adaptive scheduler and rescaling API provide powerful capabilities to increase or decrease parallelism. The main benefit I understand of d

Re: [ANNOUNCE] Apache Flink 2.0 Preview released

Hi Benoit, Please find the result here[1]. The Nexmark repo[2] does not officially support the flink 2.0 preview version. However, we have made a PR[3] for this and once it is merged, we will offer a guide to run Nexmark Q20 with disaggregated state management. [1] https://github.com/ververica/

Re: [DISCUSS] Is it a bug that the AdaptiveScheduler does not prioritize releasing TaskManagers during downscaling in Application mode?

> Is it considered an error if the adaptive scheduler fails to release the task manager during scaling? +1 . When we enable adaptive mode and perform scaling operations on tasks, a significant part of the goal is to reduce resource usage for the tasks. However, due to some logic in the adaptive sc