[jira] [Created] (FLINK-33121) Failed precondition in JobExceptionsHandler due to concurrent global failures

Panagiotis Garefalakis created FLINK-33121: -- Summary: Failed precondition in JobExceptionsHandler due to concurrent global failures Key: FLINK-33121 URL: https://issues.apache.org/jira/browse/FLINK-33121

[jira] [Created] (FLINK-33120) FLIP-361: Improve GC Metrics

Gyula Fora created FLINK-33120: -- Summary: FLIP-361: Improve GC Metrics Key: FLINK-33120 URL: https://issues.apache.org/jira/browse/FLINK-33120 Project: Flink Issue Type: Improvement Co

[RESULT][VOTE] FLIP-361: Improve GC Metrics

Hi Everyone, The proposal, FLIP-361: Improve GC Metrics, has been unanimously accepted with 15 votes (8 binding) . +1 votes: - Gyula Fora (binding) - Rui Fan (binding) - Ahmed Hamdy - Chen Zhanghao - Conrad Jam - Samrat Deb - Xintong Song (binding) - Matt Wang - Dong Lin (binding) - Ve

Re: [VOTE] FLIP-361: Improve GC Metrics

Thank you all! Closing this vote On Mon, Sep 18, 2023 at 6:16 PM Jing Ge wrote: > looks great, thanks! > +1(binding) > > Best regards, > Jing > > On Mon, Sep 18, 2023 at 8:11 AM Yuepeng Pan wrote: > > > +1 (non-binding). > > > > > > Best, > > Yuepeng. > > > > > > > > > > > > 在 2023-09-16 12:52:

Re: Close orphaned/stale PRs

Thanks for your response, Martijn. > What's the added value of closing these PRs It mainly helps the project maintainers/reviewers to focus on only the actively updated trimmed list of PRs that are ready for review. It helps Flink users who are waiting on a PR that enhances an existing feature o

Re: [DISCUSS] FLIP-368 Reorganize the exceptions thrown in state interfaces

Hi Zakelly, Thanks for bringing this up. +1 for reorganizing. IIUC, this proposal aims to change all state-related exceptions to unchecked exceptions. If users have caught checked exceptions (such as IOException ) in their code, leaving the code as is would also work. Is it possible not to put a

[jira] [Created] (FLINK-33119) The pojo result returned be procedure should be Row of fields in the pojo instead of the whole pojo object

luoyuxia created FLINK-33119: Summary: The pojo result returned be procedure should be Row of fields in the pojo instead of the whole pojo object Key: FLINK-33119 URL: https://issues.apache.org/jira/browse/FLINK-33119

[Discuss] FLIP-366: Support standard YAML for FLINK configuration

Hi devs, I would like to start a discussion about FLIP-366: Support standard YAML for FLINK configuration[1] The current flink-conf.yaml parser in FLINK is not a standard YAML parser, which has some shortcomings. Firstly, it does not support nested structure configuration items and only supports

Re: [Discuss] CRD for flink sql gateway in the flink k8s operator

Thanks for all the valuable feedback, helm charts / templates sound good to me too. It will bring a lot of convenience to the production deployment of Sql-Gateway, looking forward to this, thanks Best, Shammon FY On Wed, Sep 20, 2023 at 10:01 AM Yangze Guo wrote: > Thanks for the reply, Gyula.

[jira] [Created] (FLINK-33118) Remove the PythonBridgeUtils

Jiang Xin created FLINK-33118: - Summary: Remove the PythonBridgeUtils Key: FLINK-33118 URL: https://issues.apache.org/jira/browse/FLINK-33118 Project: Flink Issue Type: Improvement Comp

Re: [Discuss] CRD for flink sql gateway in the flink k8s operator

Thanks for the reply, Gyula. Also thanks for the input from Thomas and Dongwoo. Helm charts / templates for creating SQL Gateway deployment / services sounds good to me. I'll work on it and also update that to the OLAP quickstart doc. Best, Yangze Guo On Tue, Sep 19, 2023 at 11:46 PM Dongwoo Kim

Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

Thank you for the update, the FLIP now looks good to me. Chen Zhanghao 于2023年9月19日周二 22:50写道： > > Thanks to everyone for the valuable inputs, we learnt a lot during the > discussion. We've updated the FLIP in three main aspects based on the > discussion here: > > - Add a new subsection on keepi

Re: Close orphaned/stale PRs

Hi Venkata and Martijn, Stale PRs can clutter the repository, and closing them helps keep it organized and ensures that only relevant and up-to-date PRs are present. An open PR should imply that work is ongoing or needs review. On the other hand, a PR that reaches a stale state rather means the pr

Re: [DISCUSS] [FLINK-32873] Add a config to allow disabling Query hints

Hey Jark, Sounds fine to me. @Bonnie does that also work for you? Best regards, Martijn On Fri, Sep 15, 2023 at 1:28 PM Jark Wu wrote: > > Hi Martijn, > > Thanks for the investigation. I found the blog[1] shows a use case > that they use "OPTIMIZER_IGNORE_HINTS" to check if embedded > hints ca

[RESULT][VOTE] FLIP-323: Support Attached Execution on Flink Application Completion for Batch Jobs

Hi Everyone, Thank you for the review and votes, am happy to announce that FLIP-323 has been accepted. There are 9 votes, 3 binding and 6 nonbinding, no disap

Re: Close orphaned/stale PRs

Hi Venkata, Thanks for opening the discussion, I've been thinking about it quite a bit but I'm not sure what's the right approach. >From your proposal, the question would be "What's the added value of closing these PRs"? I don't see an immediate value of that: it would just close PRs where mainta

Close orphaned/stale PRs

Hi Flink devs, There are currently over 1,000 open pull requests (PRs) in the Apache Flink repository, with only 162 having been updated in the last two months

Re: Future of classical remoting in Pekko

I think that is totally fine, because any Pekko related changes can only be added to the first patch release of 1.18 at this point, as there is an RC0 [1] already so the release process will be initiated soon. I am glad the mentioned PR got merged, did not have the chance to review. [1] https:/

Re: [ANNOUNCE] Apache Flink Stateful Functions Release 3.3.0 released

Thanks Martijn! On Tue, Sep 19, 2023 at 6:38 AM Martijn Visser wrote: > The Apache Flink community is excited to announce the release of > Stateful Functions 3.3.0! > > Stateful Functions is a cross-platform stack for building Stateful > Serverless applications, making it radically simpler to de

FW: RE: [DISCUSS] FLIP-314: Support Customized Job Lineage Listener

Hi there, I am looking at the interfaces. If I am reading it correctly,there is one relationship between the source and sink and this relationship represents the operational lineage. Lineage is usually represented as asset -> process - > asset – see for example https://egeria-project.org/featur

Re: [Discuss] CRD for flink sql gateway in the flink k8s operator

A simple Helm chart that covers both scenarios - running gateway 1) as a sidecar, 2) as an independent deployment- with solid docs sounds good to me. It would be nice if we could also include optional features in the chart such as k8s service for exposing the sql gateway. Best regards, Dongwoo 20

RE: [DISCUSS] FLIP-314: Support Customized Job Lineage Listener

Hi, I notice that there is an experimental lineage integration for Flink with OpenLineage https://openlineage.io/docs/integrations/flink . I think this feature would allow for a superior Flink OpenLineage integration, Kind regards, David. From: XTransfer Date: Tuesday, 19 September 2023

回复: [Discuss] FLIP-362: Support minimum resource limitation

Thanks for driving this, Xiangyu. We use Session clusters for quick SQL debugging internally, and found cold-start job submission slow due to lack of the exact minimum resource reservation feature proposed here. This should improve the experience a lot for running short lived-jobs in session clu

回复: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

Thanks to everyone for the valuable inputs, we learnt a lot during the discussion. We've updated the FLIP in three main aspects based on the discussion here: - ​Add a new subsection on keeping downstream operators' parallelism unchanged by wrapping the source transformation in a phantom transfo

Re: [DISCUSS] FLIP-314: Support Customized Job Lineage Listener

Thanks Shammon for this proposal. That’s helpful for collecting the lineage of Flink tasks. Looking forward to its implementation. Best, Jiabao > 2023年9月18日 20:56，Leonard Xu 写道： > > Thanks Shammon for the informations, the comment makes the lifecycle clearer. > +1 > > > Best, > Leonard >

Re: [DISCUSS] FLIP-327: Support stream-batch unified operator to improve job throughput when processing backlog data

Hi all, Thank you for the comments! If there is no further comment, we will open the voting thread in 3 days. Cheers, Xuannan and Dong

Re: [Discuss] CRD for flink sql gateway in the flink k8s operator

It is already possible to bring up a SQL Gateway as a sidecar utilizing the pod templates - I tend to also see this more of a documentation/example issue rather than something that calls for a separate CRD or other dedicated operator support. Thanks, Thomas On Tue, Sep 19, 2023 at 3:41 PM Gyula

Re: [Discuss] CRD for flink sql gateway in the flink k8s operator

Based on this I think we should start with simple Helm charts / templates for creating the `FlinkDeployment` together with a separate Deployment for the SQL Gateway. If the gateway itself doesn't integrate well with the operator managed CRs (sessionjobs) then I think it's better and simpler to have

Re: [VOTE] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

+1(binding) Best, Weihua On Tue, Sep 19, 2023 at 6:16 PM Jing Ge wrote: > +1(binding) Thanks! > > Best regards, > Jing > > On Tue, Sep 19, 2023 at 9:01 AM Chen Zhanghao > wrote: > > > Hi Devs, > > > > Thanks for all the feedbacks on FLIP-363: Unify the Representation of > > TaskManager Locati

[CVE-2023-41834] Apache Flink Stateful Functions allowed HTTP header injection due to Improper Neutralization of CRLF Sequences

CVE-2023-41834: Apache Flink Stateful Functions allowed HTTP header injection due to Improper Neutralization of CRLF Sequences Severity: moderate Vendor: The Apache Software Foundation Versions Affected: Stateful Functions 3.1.0 to 3.2.0 Description: Improper Neutralization of CRLF Sequences in

[ANNOUNCE] Apache Flink Stateful Functions Release 3.3.0 released

The Apache Flink community is excited to announce the release of Stateful Functions 3.3.0! Stateful Functions is a cross-platform stack for building Stateful Serverless applications, making it radically simpler to develop scalable, consistent, and elastic distributed applications. This new release

Re: [DISCUSS] FLIP-368 Reorganize the exceptions thrown in state interfaces

Hi, Zakelly. Thanks for the proposal. +1 for reorganizing exceptions of state interfaces which indeed confuses me currently. >From my experience, users usually omit these exceptions because they cannot do much even if they catch the exceptions. I have some problems and suggestions, PTAL: 1.

Re: [VOTE] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

+1(binding) Thanks! Best regards, Jing On Tue, Sep 19, 2023 at 9:01 AM Chen Zhanghao wrote: > Hi Devs, > > Thanks for all the feedbacks on FLIP-363: Unify the Representation of > TaskManager Location in REST API and Web UI [1][2]. Given that the > consensus on the naming issue has been reached

Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

Hi Zhanghao, Thanks for your effort! Best regards, Jing On Tue, Sep 19, 2023 at 8:52 AM Chen Zhanghao wrote: > Thanks to everyone who discussed here, I really appreciate it. I've > updated the FLIP to change all occurrences of "location" to "endpoint" > instead. Given that we've reached consen

[RESULT][VOTE] Apache Flink Stateful Functions Release 3.3.0, release candidate #2

I'm happy to announce that we have unanimously approved this release. There are 3 approving votes, 3 of which are binding: * Robert * Danny * Martijn There are no disapproving votes. I'll work on completing the release. Thanks all! Best regards, Martijn

[jira] [Created] (FLINK-33117) a wrong scala example in udfs page

cuiyanxiang created FLINK-33117: --- Summary: a wrong scala example in udfs page Key: FLINK-33117 URL: https://issues.apache.org/jira/browse/FLINK-33117 Project: Flink Issue Type: Bug Co

[jira] [Created] (FLINK-33116) CliClientTest.testCancelExecutionInteractiveMode fails with NPE on AZP

Sergey Nuyanzin created FLINK-33116: --- Summary: CliClientTest.testCancelExecutionInteractiveMode fails with NPE on AZP Key: FLINK-33116 URL: https://issues.apache.org/jira/browse/FLINK-33116 Project:

[jira] [Created] (FLINK-33115) AbstractHadoopRecoverableWriterITCase is hanging with timeout on AZP

Sergey Nuyanzin created FLINK-33115: --- Summary: AbstractHadoopRecoverableWriterITCase is hanging with timeout on AZP Key: FLINK-33115 URL: https://issues.apache.org/jira/browse/FLINK-33115 Project: F

Re: [ANNOUNCE] Release 1.18.0, release candidate #0

Hi Yuan and Jing, Thank you for sharing your thoughts. I completely agree that it is our top priority to ensure that there are no regressions from the last commit the previous benchmark pipeline covered to the final commit of this release. I will try to get this result first. Best, Zakelly On T

[jira] [Created] (FLINK-33114) RightOuterJoinTaskTest.testCancelRightOuterJoinTaskWhileBuilding:232 Task thread did not finish within 60 seconds

Sergey Nuyanzin created FLINK-33114: --- Summary: RightOuterJoinTaskTest.testCancelRightOuterJoinTaskWhileBuilding:232 Task thread did not finish within 60 seconds Key: FLINK-33114 URL: https://issues.apache.org/j

[DISCUSS] FLIP-368 Reorganize the exceptions thrown in state interfaces

Hi everyone, I would like to initiate a discussion on FLIP-368, which focuses on reorganizing the exceptions thrown in state interfaces [1]. Currently, we have identified several problems with the exceptions thrown by state-related interfaces: 1. The exception types thrown by each interface are

Re: [ANNOUNCE] Release 1.18.0, release candidate #0

Hi Thanks Zakelly and Yuan for your effort and update. Since we changed the hardware, IMHO, if we are able to reach a consensus in the community that there is no regression with the benchmarks, we could consider releasing rc1 without waiting for the new baseline scores which might take days. Best

Re: [ANNOUNCE] Release 1.18.0, release candidate #0

Hey Zakelly, Thanks very much for the efforts to re-build the entire benchmark environment. As long as we have 1) the pipeline set up and ready (no need for the entire portal ready), 2) get benchmark comparison numbers (comparing with the commit just before the benchmark pipeline is down) and 3)

[jira] [Created] (FLINK-33113) YARNSessionFIFOITCase.checkForProhibitedLogContents: Interrupted waiting to send RPC request to server

Sergey Nuyanzin created FLINK-33113: --- Summary: YARNSessionFIFOITCase.checkForProhibitedLogContents: Interrupted waiting to send RPC request to server Key: FLINK-33113 URL: https://issues.apache.org/jira/browse/

Re: [ANNOUNCE] Release 1.18.0, release candidate #0

Hi everyone, I am working on rebuilding the benchmark pipeline and it's almost done. However, due to the change in machines for benchmarking, I will need a few more days to run tests and gather the baseline scores for further comparison. Once the pipeline is fully ready, we will proceed with the p

Re: [VOTE] Apache Flink Stateful Functions Release 3.3.0, release candidate #2

Hi everyone, Thanks for your votes, the voting is now closed. I'll announce the results in a separate thread. Best regards, Martijn On Mon, Sep 18, 2023 at 4:43 PM Danny Cranmer wrote: > +1 (binding) > > - Contents of dist matches 3.2.0 > - Checksum/sigs of dist and maven repo match > - Sourc

Re: [DISCUSS] FLIP-327: Support stream-batch unified operator to improve job throughput when processing backlog data

Thanks for all of the updates! I'm looking forward to trying out those improvements. Best, Piotrek pon., 18 wrz 2023 o 09:33 Xintong Song napisał(a): > Thanks for addressing my comments. > > LGTM > > Best, > > Xintong > > > > On Mon, Sep 18, 2023 at 3:10 PM Dong Lin wrote: > > > Hi Xintong, >

回复: [VOTE] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

Hi Devs, Thanks for all the feedbacks on FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI [1][2]. Given that the consensus on the naming issue has been reached (using "endpoint" instead of "location"), I'd like to restart the vote for it. The vote will be open