[DISCUSS] Flink PubSub connector bump release needed

2024-11-07 Thread Radek Stankiewicz
Dear Flink devs, I would like to initiate a discussion about the need for a new release flink-connector-gcp-pubsub. I found and fixed a bug (FLINK-36616) that was blocking a project and as an interim step we can release the package internally but to provide benefit to everyone it would be great if

Re: [VOTE]FLIP-480: Support to deploy script in application mode

2024-11-07 Thread Lincoln Lee
+1 (binding) Best, Lincoln Lee Shengkai Fang 于2024年11月8日周五 15:11写道: > Hi everyone, > > I'd like to start a vote on FLIP-480: Support to deploy script in > application mode[1]. The discussion can be found here[2]. > > The vote will be open for at least 72 hours unless there are any objections

Re: [DISCUSS] Changing return type of window properties (WINDOW_START/END)

2024-11-07 Thread Leonard Xu
Thanks Dawid for bringing this ticket to dev mailing list and Timo’s kindly ping. Firstly the example result is a little strange for me too, the print window_time looks incorrect, Could you post your entire example especially your session time zone? Back to the window_start/end return type, bo

[VOTE]FLIP-480: Support to deploy script in application mode

2024-11-07 Thread Shengkai Fang
Hi everyone, I'd like to start a vote on FLIP-480: Support to deploy script in application mode[1]. The discussion can be found here[2]. The vote will be open for at least 72 hours unless there are any objections or insufficient votes. Best, Shengkai [1] https://cwiki.apache.org/confluence/disp

[jira] [Created] (FLINK-36674) Flink will not commit if checkpointing is not enabled

2024-11-07 Thread slankka (Jira)
slankka created FLINK-36674: --- Summary: Flink will not commit if checkpointing is not enabled Key: FLINK-36674 URL: https://issues.apache.org/jira/browse/FLINK-36674 Project: Flink Issue Type: Bug

[DISCUSS] Feature Freeze for Flink 2.0 - Jan 13

2024-11-07 Thread Xintong Song
Hi devs, After an offline discussion between the release managers, we'd like to propose *Jan 13 (Mon), 2025 as the feature freeze date for Flink 2.0, *for the following reasons: - Flink 2.0 Preview was released on Oct 23, ~3 weeks later than originally planned. To leave enough time for tryi

Re: [VOTE] Release flink-connector-aws v5.0.0, release candidate #3

2024-11-07 Thread Xintong Song
+1 (binding) - verified signatures and checksum - built from source - reviewed web PR Best, Xintong On Fri, Nov 8, 2024 at 9:44 AM weijie guo wrote: > +1(binding) > > - verified signatures > - verified checksums > - checked github release tag > - checked release notes > - reviewed the web P

Re: [VOTE] Release flink-connector-aws v5.0.0, release candidate #3

2024-11-07 Thread weijie guo
+1(binding) - verified signatures - verified checksums - checked github release tag - checked release notes - reviewed the web PR - built from source Best regards, Weijie Danny Cranmer 于2024年11月7日周四 18:25写道: > Thanks Hong for driving and addressing the issues in the previous release > candid

[jira] [Created] (FLINK-36673) Operator is not properly handling failed deployments without savepoints

2024-11-07 Thread Yaroslav Tkachenko (Jira)
Yaroslav Tkachenko created FLINK-36673: -- Summary: Operator is not properly handling failed deployments without savepoints Key: FLINK-36673 URL: https://issues.apache.org/jira/browse/FLINK-36673 P

RE: Flink technical debt.

2024-11-07 Thread Nic Townsend
Hi Robert, David I am very much interested in a backlog review call – personally for two reasons: 1. Learning a bit more about the state of Flink, understanding how each issue would be triaged (and understanding what/why the severity is) 2. Seeing if there are any obvious places that I ca

[DISCUSS] FLIP-481: Introduce Event Reporting

2024-11-07 Thread Piotr Nowojski
Hi all! I would like to open up for the discussion a new FLIP [1] Motivation Currently, Flink observability has support for Metrics and Traces. We suggest enhancing the observability capabilities by adding support for Events. This can be used to track the most important events that happen in Fli

[jira] [Created] (FLINK-36672) Kinesis Table API for new KDS source should infer region from streamARN

2024-11-07 Thread Hong Liang Teoh (Jira)
Hong Liang Teoh created FLINK-36672: --- Summary: Kinesis Table API for new KDS source should infer region from streamARN Key: FLINK-36672 URL: https://issues.apache.org/jira/browse/FLINK-36672 Project

Flink V1 to V2 state migration

2024-11-07 Thread David Radley
Hello, At Flink Forward I learnt that Fink V1 state could not be migrated to V2. I think this would be a big migration inhibitor for current Flink users, as they would need to throw away their existing state. As such I think this is a critical possible blocking issue. Prior to Flink 2 going out,

Re: Micro batching with flink

2024-11-07 Thread Anil Dasari
Just following up—please let me know if any of you have recommendations for implementing the mentioned use case. Thanks.On Tuesday, October 29, 2024 at 10:37:30 PM PDT, Anil Dasari wrote: Hi Venkat,Thanks for the reply.  Microbatching is a data processing technique where small batche

RE: Re: Flink technical debt.

2024-11-07 Thread Ammu P
Hi David, Thanks for this initiative. It looks really helpful and please count me in. Regards, Ammu Parvathy On 2024/11/07 10:35:43 Hong Liang wrote: > Hi David, > > Thanks for this proposal. I agree that this is something we can strive to > do better in the Flink community, and I would be keen

RE: [DISCUSS] FLIP-440: User-defined SQL operators / ProcessTableFunction (PTF)

2024-11-07 Thread David Radley
Hi Timo, Thank you for the well structured and referenced Flip. This looks really useful. Some thoughts: 1. I wonder what class of scenarios that PTF’s do not do well in and that DataStream API would be preferable/required? Assuming we have PTF Map and List support. 2. I see /** *

[jira] [Created] (FLINK-36671) Kinesis Table API for new KDS source does not support metadata.

2024-11-07 Thread Hong Liang Teoh (Jira)
Hong Liang Teoh created FLINK-36671: --- Summary: Kinesis Table API for new KDS source does not support metadata. Key: FLINK-36671 URL: https://issues.apache.org/jira/browse/FLINK-36671 Project: Flink

FLIP-484: Add custom metric variables to operators

2024-11-07 Thread Piotr Nowojski
Hi all! I would like to open up for discussion a new FLIP-484 [1]. Motivation When running a Flink job that reads data from multiple sources and writes to multiple sinks, it would be helpful to group metrics from each source/sink instance together, for example based on the underlying name of the

[DISCUSS] FLIP-484: Add custom metric variables to operators

2024-11-07 Thread Piotr Nowojski
czw., 7 lis 2024 o 14:44 Piotr Nowojski napisał(a): > Hi all! > > I would like to open up for discussion a new FLIP-484 [1]. > > Motivation > When running a Flink job that reads data from multiple sources and writes > to multiple sinks, it would be helpful to group metrics from each > source/sink

[DISCUSS] FLIP-482: Add OpenTelemetryEventReporter

2024-11-07 Thread Piotr Nowojski
Hi all! I would like to open up for discussion a new FLIP-482 [1]. Motivation FLIP-481 [2] is adding the EventReporter interface. However with FLIP-481 [2] alone, Log4jTraceReporter would be the only available implementation of the EventReporter interface, which is not very helpful. We propose t

[DISCUSS] FLIP-483: Add support for children Spans

2024-11-07 Thread Piotr Nowojski
Hi all! I would like to open up for discussion a new FLIP-483 [1]. Motivation FLIP-384 [2] added trace/span reporting capability to Flink, which has been used in a couple of places, like reporting checkpointing and recovery processes. With flat/childless structure of spans it is difficult to acc

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

2024-11-07 Thread Junrui Lee
Thank you so much for your kind congratulations and warm welcome! Looking forward to contributing more to the community. Best, Junrui Ron Liu 于2024年11月7日周四 17:00写道: > Congratulations Junrui > > Best, > Ron > > wenjin 于2024年11月7日周四 11:57写道: > > > Congratulations Junrui~ > > > > Best regards, >

Re: Flink technical debt.

2024-11-07 Thread Hong Liang
Hi David, Thanks for this proposal. I agree that this is something we can strive to do better in the Flink community, and I would be keen to help out here. +1 to the suggestion for a recurring working group meeting to triage and assign PRs. I think the suggestions we have on the thread are great

Re: [VOTE] Release flink-connector-aws v5.0.0, release candidate #3

2024-11-07 Thread Danny Cranmer
Thanks Hong for driving and addressing the issues in the previous release candidate. +1 (binding) * Release notes look good * Source archive signatures and checksums match * There are no binaries in the source * Build source using Maven * Verified Apache headers exist and are correct in the new S

Re: [ANNOUNCE] New Apache Flink Committer - Junrui Li

2024-11-07 Thread Ron Liu
Congratulations Junrui Best, Ron wenjin 于2024年11月7日周四 11:57写道: > Congratulations Junrui~ > > Best regards, > Wenjin > > > 2024年11月5日 19:59,Zhu Zhu 写道: > > > > Hi everyone, > > > > On behalf of the PMC, I'm happy to announce that Junrui Li has become a > > new Flink Committer! > > > > Junrui ha