[jira] [Created] (FLINK-35811) Add the documentation sections to describe the usage of users-oriented interfaces

2024-07-11 Thread RocMarshal (Jira)
RocMarshal created FLINK-35811: -- Summary: Add the documentation sections to describe the usage of users-oriented interfaces Key: FLINK-35811 URL: https://issues.apache.org/jira/browse/FLINK-35811 Project

Re: [DISCUSS] FLIP-468: Introducing StreamGraph-Based Job Submission.

2024-07-11 Thread Zhu Zhu
+1 for the FLIP. With Dataset API removed in Flink 2.0, StreamGraph will be the only operator level DAG of a Flink job. Exposing it directly to the scheduler can be helpful to deliver some exciting features. This FLIP meationed adaptive plan optimization and operator insights. I'm also thinking o

[jira] [Created] (FLINK-35812) Move tuple interfaces to flink-core-api

2024-07-11 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-35812: --- Summary: Move tuple interfaces to flink-core-api Key: FLINK-35812 URL: https://issues.apache.org/jira/browse/FLINK-35812 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-35813) Transform schema operator failed to trigger checkpoint with bounded source

2024-07-11 Thread yux (Jira)
yux created FLINK-35813: --- Summary: Transform schema operator failed to trigger checkpoint with bounded source Key: FLINK-35813 URL: https://issues.apache.org/jira/browse/FLINK-35813 Project: Flink Iss

Re: [VOTE] FLIP-465: Introduce DESCRIBE FUNCTION

2024-07-11 Thread Martijn Visser
+1 (binding) On Sat, Jul 6, 2024 at 6:52 PM Yubin Li wrote: > +1 (non-binding) > > Best, > Yubin > > On Fri, Jul 5, 2024 at 4:09 AM Ferenc Csaky > wrote: > > > > +1 (non-binding) > > > > Best, > > Ferenc > > > > > > > > > > On Thursday, 4 July 2024 at 16:20, Feng Jin > wrote: > > > > > > > > >

RE: [VOTE] FLIP-465: Introduce DESCRIBE FUNCTION

2024-07-11 Thread David Radley
+1 (non-binding) Kind regards , David. From: Martijn Visser Date: Thursday, 11 July 2024 at 10:10 To: dev@flink.apache.org Subject: [EXTERNAL] Re: [VOTE] FLIP-465: Introduce DESCRIBE FUNCTION +1 (binding) On Sat, Jul 6, 2024 at 6:52 PM Yubin Li wrote: > +1 (non-binding) > > Best, > Yubin > >

Re: [DISCUSS] FLIP-468: Introducing StreamGraph-Based Job Submission.

2024-07-11 Thread Fabian Paul
Thanks for drafting this FLIP. I really like the idea of introducing a concept in Flink that is close to a logical plan submission. I have a few questions about the proposal and its future evolvability. - What is the future plan for job submissions in Flink? With the current proposal, Flink will

Re: [DISCUSS] FLIP-468: Introducing StreamGraph-Based Job Submission.

2024-07-11 Thread David Morávek
Hi Junrui, Thank you for drafting the FLIP. I really appreciate the direction it’s taking. We’ve discussed similar approaches multiple times, and it’s great to see this progress. I have a few questions and thoughts: * 1. Transformations in StreamGraphGenerator:* Should we consider taking this a

[jira] [Created] (FLINK-35814) Don't scale up continuously when the throughout cannot be increased after scaling up

2024-07-11 Thread Rui Fan (Jira)
Rui Fan created FLINK-35814: --- Summary: Don't scale up continuously when the throughout cannot be increased after scaling up Key: FLINK-35814 URL: https://issues.apache.org/jira/browse/FLINK-35814 Project: F

[jira] [Created] (FLINK-35815) KinesisProxySyncV2 doesn't always retry throttling exceptions.

2024-07-11 Thread Krzysztof Dziolak (Jira)
Krzysztof Dziolak created FLINK-35815: - Summary: KinesisProxySyncV2 doesn't always retry throttling exceptions. Key: FLINK-35815 URL: https://issues.apache.org/jira/browse/FLINK-35815 Project: Fl

[DISCUSS] FLIP-XXX: Aligning timeout logic in the AdaptiveScheduler's WaitingForResources and Executing states

2024-07-11 Thread Zdenek Tison
Hello, Our team has been working on several improvements for AdaptiveScheduler, specifically focusing on aligning logic and timeouts in the WaitingForResources and Executing states. We believe these enhancements will improve the adaptive scheduler's robustness and maintainability. For more detail

[jira] [Created] (FLINK-35816) Non-mergeable proctime tvf window aggregate needs to fallback to group aggregate

2024-07-11 Thread lincoln lee (Jira)
lincoln lee created FLINK-35816: --- Summary: Non-mergeable proctime tvf window aggregate needs to fallback to group aggregate Key: FLINK-35816 URL: https://issues.apache.org/jira/browse/FLINK-35816 Projec

Re: [VOTE] FLIP-465: Introduce DESCRIBE FUNCTION

2024-07-11 Thread Sergey Nuyanzin
+1 (binding) thanks for driving this On Thu, Jul 11, 2024 at 11:29 AM David Radley wrote: > > +1 (non-binding) > > Kind regards , David. > From: Martijn Visser > Date: Thursday, 11 July 2024 at 10:10 > To: dev@flink.apache.org > Subject: [EXTERNAL] Re: [VOTE] FLIP-465: Introduce DESCRIBE FUNCT

Re: [VOTE] FLIP-465: Introduce DESCRIBE FUNCTION

2024-07-11 Thread Timo Walther
+1 (binding) Thanks, Timo On 11.07.24 15:43, Sergey Nuyanzin wrote: +1 (binding) thanks for driving this On Thu, Jul 11, 2024 at 11:29 AM David Radley wrote: +1 (non-binding) Kind regards , David. From: Martijn Visser Date: Thursday, 11 July 2024 at 10:10 To: dev@flink.apache.org Subjec

[jira] [Created] (FLINK-35817) High RSS Memory Growth in Long-Running Flink Tasks with Low Heap Usage: Is Prometheus Metrics Reporting the Culprit?

2024-07-11 Thread ZHangTianLong (Jira)
ZHangTianLong created FLINK-35817: - Summary: High RSS Memory Growth in Long-Running Flink Tasks with Low Heap Usage: Is Prometheus Metrics Reporting the Culprit? Key: FLINK-35817 URL: https://issues.apache.org/jir

[jira] [Created] (FLINK-35818) Flink operator unable to be installed if ResourceQuota's are defined on cluster

2024-07-11 Thread Tony Garrard (Jira)
Tony Garrard created FLINK-35818: Summary: Flink operator unable to be installed if ResourceQuota's are defined on cluster Key: FLINK-35818 URL: https://issues.apache.org/jira/browse/FLINK-35818 Proje

[jira] [Created] (FLINK-35819) Upgrade to AWS SDK 2.26.19

2024-07-11 Thread Burak Ozakinci (Jira)
Burak Ozakinci created FLINK-35819: -- Summary: Upgrade to AWS SDK 2.26.19 Key: FLINK-35819 URL: https://issues.apache.org/jira/browse/FLINK-35819 Project: Flink Issue Type: Sub-task

failed to submit Job to Flink standalone ZooKeeper-HA-cluster

2024-07-11 Thread love_h1...@126.com
I have a standalone ZooKeeper-HA-cluster. First restart ZK cluster. Second Submit 31 Jobs with Command-Line , `./bin/flink run` Then I caught an exception. Specifically, there were 4 jobs that failed to submit. ``` 18:55:20,243 - [Flink-DispatcherRestEndpoint-thread-2] - [WARN ] - [org.apache.f

[jira] [Created] (FLINK-35820) Converting Duration to String fails for big values

2024-07-11 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-35820: Summary: Converting Duration to String fails for big values Key: FLINK-35820 URL: https://issues.apache.org/jira/browse/FLINK-35820 Project: Flink Is

[jira] [Created] (FLINK-35821) ResumeCheckpointManuallyITCase failed with File X does not exist or the user running Flink C has insufficient permissions to access it

2024-07-11 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-35821: -- Summary: ResumeCheckpointManuallyITCase failed with File X does not exist or the user running Flink C has insufficient permissions to access it Key: FLINK-35821 URL: https://i

Re: [VOTE] FLIP-465: Introduce DESCRIBE FUNCTION

2024-07-11 Thread Natea Eshetu Beshada
Thanks everyone for voting! Closing the vote now and will follow up with the results in a separate email. On Thu, Jul 11, 2024 at 6:46 AM Timo Walther wrote: > +1 (binding) > > Thanks, > Timo > > On 11.07.24 15:43, Sergey Nuyanzin wrote: > > +1 (binding) > > > > thanks for driving this > > > > O

[RESULT][VOTE] FLIP-465: Introduce DESCRIBE FUNCTION

2024-07-11 Thread Natea Eshetu Beshada
Happy to share that the vote for FLIP-465 [1] was unanimously approved! The discussion thread [2] was also mostly uncontroversial. Voting is now closed. [3] Voting included 9 votes, out of which 3 were binding -- and no disapproving votes. And Martijn voted twice haha! - Jim Hughs - Yanquan Lv -

[jira] [Created] (FLINK-35822) FLIP-465: Introduce DESCRIBE FUNCTION

2024-07-11 Thread Natea Eshetu Beshada (Jira)
Natea Eshetu Beshada created FLINK-35822: Summary: FLIP-465: Introduce DESCRIBE FUNCTION Key: FLINK-35822 URL: https://issues.apache.org/jira/browse/FLINK-35822 Project: Flink Issue T

Re: [DISCUSS] FLIP-468: Introducing StreamGraph-Based Job Submission.

2024-07-11 Thread Ron Liu
Hi, Junrui The FLIP proposal looks good to me. I have the same question as Fabian: > For join strategies, they are only applicable when using an optimizer (that's currently not part of Flink's runtime) with the Table API or Flink SQL. How do we plan to connect the optimizer with Flink's runtime?

[jira] [Created] (FLINK-35823) Introduce parameters to control the upper limit of rescale to avoid unlimited shrinkage due to server-side bottlenecks or data skew.

2024-07-11 Thread yuanfenghu (Jira)
yuanfenghu created FLINK-35823: -- Summary: Introduce parameters to control the upper limit of rescale to avoid unlimited shrinkage due to server-side bottlenecks or data skew. Key: FLINK-35823 URL: https://issues.apa

[jira] [Created] (FLINK-35824) Polish JDBC documentation : fix link and add back to top button

2024-07-11 Thread Zhongqiang Gong (Jira)
Zhongqiang Gong created FLINK-35824: --- Summary: Polish JDBC documentation : fix link and add back to top button Key: FLINK-35824 URL: https://issues.apache.org/jira/browse/FLINK-35824 Project: Flink

[jira] [Created] (FLINK-35825) HiveTableSource supports report statistics for text file

2024-07-11 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-35825: -- Summary: HiveTableSource supports report statistics for text file Key: FLINK-35825 URL: https://issues.apache.org/jira/browse/FLINK-35825 Project: Flink Issue Ty

[jira] [Created] (FLINK-35826) [SQL] Sliding window may produce unstable calculations when processing changelog data.

2024-07-11 Thread Yuan Kui (Jira)
Yuan Kui created FLINK-35826: Summary: [SQL] Sliding window may produce unstable calculations when processing changelog data. Key: FLINK-35826 URL: https://issues.apache.org/jira/browse/FLINK-35826 Projec

Re: [DISCUSS] FLIP-468: Introducing StreamGraph-Based Job Submission.

2024-07-11 Thread David Morávek
> > For batch scenario, if we want to better support dynamic plan tuning > strategies, the fundamental solution is still to put SQL Optimizer to > flink-runtime. One accompanying question is: how do you envision this to work for streaming where you need to ensure state compatibility after the pla