[jira] [Created] (FLINK-35903) Add CentralizedSlotAssigner support to adaptive scheduler

2024-07-25 Thread yuanfenghu (Jira)
yuanfenghu created FLINK-35903: -- Summary: Add CentralizedSlotAssigner support to adaptive scheduler Key: FLINK-35903 URL: https://issues.apache.org/jira/browse/FLINK-35903 Project: Flink Issue T

Re: [VOTE] FLIP-468: Introducing StreamGraph-Based Job Submission

2024-07-25 Thread Yuepeng Pan
Thanks for driving it !+1(non-binding) Best regards, Yuepeng Pan At 2024-07-26 11:49:41, "Rui Fan" <1996fan...@gmail.com> wrote: >Thanks Junrui for driving this proposal! > >+1(binding) > >Best, >Rui > >On Fri, Jul 26, 2024 at 11:02 AM Junrui Lee wrote: > >> Hi everyone, >> >> Thanks for all t

Re: [DISCUSS] FLIP-469: Supports Adaptive Optimization of StreamGraph

2024-07-25 Thread Ron Liu
Thanks for your updated, the FLIP looks good to me. Junrui Lee 于2024年7月26日周五 12:10写道: > Hi All, > > After an offline discussion with Ron, I have made the following updates to > FLIP-469: > > Introduced the OperatorsFinished class to represent the state of finished > operators, as well as the siz

[jira] [Created] (FLINK-35902) WebUI Metrics RangeError on huge parallelism

2024-07-25 Thread Sergey Paryshev (Jira)
Sergey Paryshev created FLINK-35902: --- Summary: WebUI Metrics RangeError on huge parallelism Key: FLINK-35902 URL: https://issues.apache.org/jira/browse/FLINK-35902 Project: Flink Issue Type

Re: [DISCUSS] FLIP-469: Supports Adaptive Optimization of StreamGraph

2024-07-25 Thread Junrui Lee
Hi All, After an offline discussion with Ron, I have made the following updates to FLIP-469: Introduced the OperatorsFinished class to represent the state of finished operators, as well as the size and distribution of the data they produced. The StreamGraphOptimizer strategy will now depend on Op

Re: [VOTE] FLIP-468: Introducing StreamGraph-Based Job Submission

2024-07-25 Thread Zhu Zhu
+1 (binding) Thanks, Zhu Ron Liu 于2024年7月26日周五 11:57写道: > +1(binding) > > Best, > Ron > > Rui Fan <1996fan...@gmail.com> 于2024年7月26日周五 11:50写道: > > > Thanks Junrui for driving this proposal! > > > > +1(binding) > > > > Best, > > Rui > > > > On Fri, Jul 26, 2024 at 11:02 AM Junrui Lee wrote: >

Re: [DISCUSS] FLIP-470: Support Adaptive Broadcast Join

2024-07-25 Thread Ron Liu
Hi, Xia Thanks for your reply. It looks good to me. Best, Ron Xia Sun 于2024年7月26日周五 10:49写道: > Hi Ron, > > Thanks for your feedback! > > -> creation of the join operators until runtime > > > That means when creating the AdaptiveJoinOperatorFactory, we will not > immediately create the JoinOpe

Re: [VOTE] FLIP-468: Introducing StreamGraph-Based Job Submission

2024-07-25 Thread Ron Liu
+1(binding) Best, Ron Rui Fan <1996fan...@gmail.com> 于2024年7月26日周五 11:50写道: > Thanks Junrui for driving this proposal! > > +1(binding) > > Best, > Rui > > On Fri, Jul 26, 2024 at 11:02 AM Junrui Lee wrote: > > > Hi everyone, > > > > Thanks for all the feedback about FLIP-468: Introducing Stream

Re: [VOTE] FLIP-468: Introducing StreamGraph-Based Job Submission

2024-07-25 Thread Rui Fan
Thanks Junrui for driving this proposal! +1(binding) Best, Rui On Fri, Jul 26, 2024 at 11:02 AM Junrui Lee wrote: > Hi everyone, > > Thanks for all the feedback about FLIP-468: Introducing StreamGraph-Based > Job Submission [1]. The discussion thread can be found here [2]. > > The vote will be

[VOTE] FLIP-468: Introducing StreamGraph-Based Job Submission

2024-07-25 Thread Junrui Lee
Hi everyone, Thanks for all the feedback about FLIP-468: Introducing StreamGraph-Based Job Submission [1]. The discussion thread can be found here [2]. The vote will be open for at least 72 hours unless there are any objections or insufficient votes. Best, Junrui [1] https://cwiki.apache.org/c

Re: [DISCUSS] FLIP-470: Support Adaptive Broadcast Join

2024-07-25 Thread Xia Sun
Hi Ron, Thanks for your feedback! -> creation of the join operators until runtime That means when creating the AdaptiveJoinOperatorFactory, we will not immediately create the JoinOperator. Instead, we only pass in the necessary parameters for creating the JoinOperator. The appropriate JoinOpera

Re: [DISCUSS]: Flink code generation porting to Calcite

2024-07-25 Thread Sergey Nuyanzin
thanks for the clarification given the fact how flink annotations are used across Flink repo I don't think it is a right way to not use it here as well... At the same time they are not used at runtime and I guess could be excluded I tried with "hello world" code split app and the way of declaring

[jira] [Created] (FLINK-35901) Setting web.submit.enable to false prevents FlinkSessionJobs from working

2024-07-25 Thread Ralph Blaise (Jira)
Ralph Blaise created FLINK-35901: Summary: Setting web.submit.enable to false prevents FlinkSessionJobs from working Key: FLINK-35901 URL: https://issues.apache.org/jira/browse/FLINK-35901 Project: Fl

[jira] [Created] (FLINK-35900) PyFlink: update version of Apache Beam dependency

2024-07-25 Thread Lydia L (Jira)
Lydia L created FLINK-35900: --- Summary: PyFlink: update version of Apache Beam dependency Key: FLINK-35900 URL: https://issues.apache.org/jira/browse/FLINK-35900 Project: Flink Issue Type: Bug A

Re: [DISCUSS] FLIP-XXX Apicurio-avro format

2024-07-25 Thread Danny Cranmer
Hey, Thanks for the updated FLIP. > 1.20 or beyond Given this is intended to go in the Kafka connector we can target the next Kafka connector version (v3.3.0) for Flink 1.18 and 1.19 (or 1.19 and 1.20, if 1.20 is released first). > RecordBasedSerialization::public boolean canProcess(byte[] cus

[jira] [Created] (FLINK-35899) Accumulated TumblingProcessingTimeWindows in the state for a few days

2024-07-25 Thread Sergey Anokhovskiy (Jira)
Sergey Anokhovskiy created FLINK-35899: -- Summary: Accumulated TumblingProcessingTimeWindows in the state for a few days Key: FLINK-35899 URL: https://issues.apache.org/jira/browse/FLINK-35899 Pro

[RESULT][VOTE] FLIP-466: Introduce ProcessFunction Attribute in DataStream API V2

2024-07-25 Thread Wencong Liu
The vote for FLIP-466 [1]: Introduce ProcessFunction Attribute in DataStream API V2 (discussion thread [2]) concluded. The vote[3] will be closed. There are 3 binding votes and 2 non-binding votes: Xintong Song (binding) Weijie Guo (binding) Rui Fun (binding) Li Wang (non-binding) Mingliang Liu

[jira] [Created] (FLINK-35898) When running Flink on Yarn, if the log file path contains a space, the startup will fail.

2024-07-25 Thread yiheng tang (Jira)
yiheng tang created FLINK-35898: --- Summary: When running Flink on Yarn, if the log file path contains a space, the startup will fail. Key: FLINK-35898 URL: https://issues.apache.org/jira/browse/FLINK-35898

Re: [DISCUSS] FLIP-XXX Amazon SQS Source Connector

2024-07-25 Thread Saurabh Singh
Hi Samrat, Thanks for the review and feedback. We have evaluated all the three points. Please find the answers below: 1. AWS has announced JSON protocol support in SQS [1]. Can you shed some light on how different protocols will be supported? - We will utilize the AWS client library to connect w

[jira] [Created] (FLINK-35897) Some checkpoint files and localState files can't be cleanUp when checkpoint is aborted

2024-07-25 Thread Jinzhong Li (Jira)
Jinzhong Li created FLINK-35897: --- Summary: Some checkpoint files and localState files can't be cleanUp when checkpoint is aborted Key: FLINK-35897 URL: https://issues.apache.org/jira/browse/FLINK-35897

Re: [DISCUSS] FLIP-471: Fixing watermark idleness timeout accounting

2024-07-25 Thread Piotr Nowojski
Hi Arvid, > 1. `RelativeClock` looks like a super-interface of > `org.apache.flink.util.clock.Clock`. Should we also reflect that > accordingly by extending? This should not break anything. It should be fine to do so, but to me the question is if we should do so? It doesn't give any benefit right

Re: [DISCUSS]: Flink code generation porting to Calcite

2024-07-25 Thread James Duong
Hi Sergey, I would prefer the approach of using the Flink module as a dependency so that fixes are propagated more easily. From: Sergey Nuyanzin Date: Thursday, July 25, 2024 at 12:44 AM To: dev@flink.apache.org Subject: Re: [DISCUSS]: Flink code generation porting to Calcite Hi James can you

[VOTE] Release 1.20.0, release candidate #2

2024-07-25 Thread weijie guo
Hi everyone, Please review and vote on the release candidate #2 for the version 1.20.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes

[jira] [Created] (FLINK-35896) eventHandler in RescaleApiScalingRealizer handle the event of scaling failure

2024-07-25 Thread yuanfenghu (Jira)
yuanfenghu created FLINK-35896: -- Summary: eventHandler in RescaleApiScalingRealizer handle the event of scaling failure Key: FLINK-35896 URL: https://issues.apache.org/jira/browse/FLINK-35896 Project: F

[jira] [Created] (FLINK-35895) flink parquet vectorized reader should use readNextFilteredRowGroup() instead of readNextRowGroup()

2024-07-25 Thread Kai Chen (Jira)
Kai Chen created FLINK-35895: Summary: flink parquet vectorized reader should use readNextFilteredRowGroup() instead of readNextRowGroup() Key: FLINK-35895 URL: https://issues.apache.org/jira/browse/FLINK-35895

Re: [DISCUSS]: Flink code generation porting to Calcite

2024-07-25 Thread Sergey Nuyanzin
Hi James can you please elaborate about your scenario since I feel getting lost with this. from one side the thread is called >Flink code generation porting to Calcite from another side it was mentioned >We really want to avoid having circular dependencies between Flink and Calcite. are you goin