[jira] [Created] (FLINK-35992) NettyShuffleMaster not close tiered-storage resources

xuhuang created FLINK-35992: --- Summary: NettyShuffleMaster not close tiered-storage resources Key: FLINK-35992 URL: https://issues.apache.org/jira/browse/FLINK-35992 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-35991) Resolve conflicting definitions in transform SqlFunctionTable definition

yux created FLINK-35991: --- Summary: Resolve conflicting definitions in transform SqlFunctionTable definition Key: FLINK-35991 URL: https://issues.apache.org/jira/browse/FLINK-35991 Project: Flink Issue

[jira] [Created] (FLINK-35990) Lingering Transactions with FlinkKafkaProducer after failures & scale-down

Peter Larsen created FLINK-35990: Summary: Lingering Transactions with FlinkKafkaProducer after failures & scale-down Key: FLINK-35990 URL: https://issues.apache.org/jira/browse/FLINK-35990 Project: F

Re: [DISCUSS] Remove deprecated dataset based API from State Processor API

Thanks for the feedbacks, since no objection proceeding. G On Thu, Aug 1, 2024 at 2:06 PM Gabor Somogyi wrote: > Hi Ferenc, > > That's a good point! Let's wait for Monday for further opinions. > > BR, > G > > > On Thu, Aug 1, 2024 at 12:55 PM Ferenc Csaky > wrote: > >> Hi, >> >> AFAIK 1.20 wi

Re: [DISCUSS] FLIP-468: Introducing StreamGraph-Based Job Submission.

Hi Yu Chen, Apologies for the late reply. Thanks for your feedback and for emphasizing the importance of this work for FLINK-33230. Looking forward to your contributions. Best, Junrui Junrui Lee 于 2024年8月6日周二 22:05写道： > Hi everyone, > > David, Zhu and I had an offline discussion about FLIP-468

Re: [DISCUSS] FLIP-468: Introducing StreamGraph-Based Job Submission.

Hi everyone, David, Zhu and I had an offline discussion about FLIP-468 and more. Here are the key points: 1. Flink lacks a stable public REST API for job submission. Such a public REST API will allow users to create and store compiled plans of jobs, and to directly submit/resumit these plans. It c

Re: [VOTE] FLIP-468: Introducing StreamGraph-Based Job Submission

Hi everyone, David, Zhu and I had an offline discussion about FLIP-468 and more. Here are the key points: 1. Flink lacks a stable public REST API for job submission. Such a public REST API will allow users to create and store compiled plans of jobs, and to directly submit/resumit these plans. It c

Re: [DISCUSS] FLIP-473: Introduce New SQL Operators Based on Asynchronous State APIs

Hi Xuyang, Thanks for driving this! I'm happy to see operator implementation on top of the async state. Overall it looks good, I have one minor question: It seems the only difference between the new operator implementation and the original one is the way of state accessing. Does this mean the sta

Stateful AsyncIO operator

Hi All, I have a use case where I read from Kafkf and have some records that I want to submit from Flink to a rest service with throttling, I thought that the AsyncIO operator would be the best for such use cases, however I wanted to know if AsyncIO is a stateful operator? What I want to

Re: [DISCUSS] FLIP-455: Declare async state processing and checkpoint the in-flight requests

Hi Piotr, Thanks for your explanation! Now I roughly get your point. All in all, I would be hesitant to allow users to declare callbacks > everywhere, > as it might lead to some unforeseen problems or readability issues? > I don't see a good motivation for them neither in `open`, `setup` nor > `s

[jira] [Created] (FLINK-35989) Document and log error on partially failed requests for AWS sinks

Hong Liang Teoh created FLINK-35989: --- Summary: Document and log error on partially failed requests for AWS sinks Key: FLINK-35989 URL: https://issues.apache.org/jira/browse/FLINK-35989 Project: Flin

Re: [DISCUSS] FLIP-455: Declare async state processing and checkpoint the in-flight requests

Hi again, Thanks for further explanation! > I briefly revisit all SQL operator implementations, and it seems feasible > for all scenarios. I'm not sure if this answers your question. I think you misunderstood what I meant. Initially I was wondering if we should allow for those async callbacks to

Re: [DISCUSS] FLIP-455: Declare async state processing and checkpoint the in-flight requests

Hi Piotr, Thanks for your reply! Because the remote state backend/async state accesses are only supported > on the keyed state? Yes. And since we only have a heap implementation of Operator State, offloading them to async threads seems pointless. Currently we keep the interface in sync-style a

Re: [DISCUSS] FLIP-455: Declare async state processing and checkpoint the in-flight requests

Hi Zakelly, Thanks for your responses! > IIUC, the AsyncWaitOperator and AsyncScalarFunction are all non-keyed, so > we don't need to worry too much Because the remote state backend/async state accesses are only supported on the keyed state? > Furthermore, as I mentioned, we currently > do not

[DISCUSS] Flink 2.0 Release Plan

Hi devs, It has been more than 15 months since we kicked off the 2.0 release plan[1]. With Flink 1.20 released last week, we finally entered the 2.0 release cycle. As previously decided, Becket, Jark, Martijn and I will serve as the release managers. We had a brief offline discussion (except for

[jira] [Created] (FLINK-35988) Reduce the number of state queries in the AppendOnlyFirstNFunction.

luolei created FLINK-35988: -- Summary: Reduce the number of state queries in the AppendOnlyFirstNFunction. Key: FLINK-35988 URL: https://issues.apache.org/jira/browse/FLINK-35988 Project: Flink Issu