[jira] [Created] (FLINK-34410) Disable nightly trigger in forks

Matthias Pohl created FLINK-34410: - Summary: Disable nightly trigger in forks Key: FLINK-34410 URL: https://issues.apache.org/jira/browse/FLINK-34410 Project: Flink Issue Type: Technical Debt

Re: [DISCUSS] FLIP 411: Chaining-agnostic Operator ID generation for improved state compatibility on parallelism change

Hey > AFAIK, there's no way to set UIDs for a SQL job, AFAIK you can't set UID manually, but Flink SQL generates a compiled plan of a query with embedded UIDs. As I understand it, using a compiled plan is the preferred (only?) way for Flink SQL if one wants to make any changes to query later on

Re: [DISCUSS] Alternative way of posting FLIPs

+1 for the first option as well Best, Piotrek śr., 7 lut 2024 o 16:48 Matthias Pohl napisał(a): > +1 for option 1 since it's a reasonable temporary workaround > > Moving to GitHub discussions would either mean moving the current FLIP > collection or having the FLIPs in two locations. Both optio

Re: FW: RE: [DISCUSS] FLIP-314: Support Customized Job Lineage Listener

Hi devs, According to the online-discussion in FLINK-3127 [1] and offline-discussion with Maciej Obuchowski and Zhenqiu Huang, we would like to update the lineage vertex relevant interfaces in FLIP-314 [2] as follows: 1. Introduce `LineageDataset` which represents source and sink in `LineageVerte

Re: [DISCUSS] FLIP 411: Chaining-agnostic Operator ID generation for improved state compatibility on parallelism change

Hi Chesnay, AFAIK, there's no way to set UIDs for a SQL job, it'll be great if you can share how you allow UID setting for SQL jobs. We've explored providing a visualized DAG editor for SQL jobs that allows UID setting on our internal platform, but most users found it too complicated to use. An

Re: Impact of redacting UPDATE_BEFORE fields?

Hey Kevin, In my experience it mostly depends on the type of your sinks. If all of your sinks can leverage primary keys and support upsert semantics, you don't really need UPDATE_BEFOREs altogether (you can even filter them out). But if you have sinks with append-only semantics (OR if you don't ha

Impact of redacting UPDATE_BEFORE fields?

Hi there! I have a question about Changelog Stream Processing with Flink SQL and the Flink Table API. I would like to better understand how UPDATE_BEFORE fields are used by Flink. Our team uses Debezium to extract Change Data Capture events from MySQL databases. We currently redact the `before` f

Re: Re: [VOTE] FLIP-417: Expose JobManagerOperatorMetrics via REST API

Hi Voters, JFYI, I have modified the proposed REST API path and added changes to the metric scope configuration--you can find the reasoning and discussion in the `[DISCUSS]` thread and FLIP doc. Please let me know if there are any concerns. Best, Mason On Mon, Jan 29, 2024 at 5:32 AM Thomas Weis

Re: [DISCUSS] FLIP 411: Chaining-agnostic Operator ID generation for improved state compatibility on parallelism change

The FLIP is a bit weird to be honest. It only applies in cases where users haven't set uids, but that goes against best-practices and as far as I'm told SQL also sets UIDs everywhere. I'm wondering if this is really worth the effort. On 07/02/2024 10:23, Zhanghao Chen wrote: After offline dis

Re: [DISCUSS] Alternative way of posting FLIPs

+1 for option 1 since it's a reasonable temporary workaround Moving to GitHub discussions would either mean moving the current FLIP collection or having the FLIPs in two locations. Both options do not seem to be optimal. Another concern I had was that GitHub Discussions wouldn't allow integrating

[jira] [Created] (FLINK-34409) Increase test coverage for AdaptiveScheduler

Matthias Pohl created FLINK-34409: - Summary: Increase test coverage for AdaptiveScheduler Key: FLINK-34409 URL: https://issues.apache.org/jira/browse/FLINK-34409 Project: Flink Issue Type: Te

Re: [DISCUSS] Kubernetes Operator 1.8.0 release planning

I can volunteer to be a release manager. I haven't done it for Apache/Flink or the operator before so I may be a good candidate. Ryan van Huuksloot Sr. Production Engineer | Streaming Platform [image: Shopify] On Wed, Feb

Re: jira permission request

Hi, In order to contribute to Apache Flink, please ping the author of the Jira ticket that you would like to work on, so it can be assigned to you. There are no other permissions required. Best regards, Martijn On Tue, Feb 6, 2024 at 2:25 AM 李游 wrote: > > HI, > I want to contribute to Apache f

Re: Confluence access request

Hi, Sorry for the late reply, but this has been recently been disabled by the ASF. There's an open discussion thread at https://lists.apache.org/thread/rkpvlnwj9gv1hvx1dyklx6k88qpnvk2t on how to deal with this. Best regards, Martijn On Tue, Jan 30, 2024 at 10:16 AM tanjialiang wrote: > > Hi, d

Re: Confluence access

Hi, Sorry for the late reply, but this has been recently been disabled by the ASF. There's an open discussion thread at https://lists.apache.org/thread/rkpvlnwj9gv1hvx1dyklx6k88qpnvk2t on how to deal with this. Best regards, Martijn On Tue, Jan 23, 2024 at 4:03 AM jufang he wrote: > > Hi devs!

Re: [DISCUSS] Alternative way of posting FLIPs

Thanks Martijn moving this forward! +1 for the first solution, because as of now it looks like this is a temporary solution and we're still looking forward to the improvement by ASF Infra, when the access is ok for contributors, we can back to the current workflow. For solution 2, one visible dow

[DISCUSS] Alternative way of posting FLIPs

Hi all, ASF Infra has confirmed to me that only ASF committers can access the ASF Confluence site since a recent change. One of the results of this decision is that users can't signup and access Confluence, so only committers+ can create FLIPs. ASF Infra hopes to improve this situation when they

[ANNOUNCE] Apache flink-connector-kafka v3.1.0 released

The Apache Flink community is very happy to announce the release of Apache flink-connector-kafka v3.1.0. This release is compatible with Apache Flink 1.17 and 1.18. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data str

[RESULT][VOTE] Release flink-connector-kafka v3.1.0, release candidate #1

I'm happy to announce that we have unanimously approved this release. There are 5 approving votes, 3 of which are binding: * Hang Ruan (non-binding) * Mason Chen (non-binding) * Qingsheng Ren (binding) * Maximilian Michels (binding) * Martijn Visser (binding) There are no disapproving votes. I'v

[jira] [Created] (FLINK-34408) VeryBigPbProtoToRowTest#testSimple fails with OOM

Matthias Pohl created FLINK-34408: - Summary: VeryBigPbProtoToRowTest#testSimple fails with OOM Key: FLINK-34408 URL: https://issues.apache.org/jira/browse/FLINK-34408 Project: Flink Issue Typ

Re: Flink jdbc connector rc3 for flink 1.18

Hi David, Thanks for testing. Yes the jars are built from the same sources and same git tag apart from the Flink version. as it was mentioned in jdbc connector RC thread [1] >The complete staging area is available for your review, which includes: >* all artifacts to be deployed to the Maven Cen

Flink jdbc connector rc3 for flink 1.18

Hi , I had a question on Flink jdbc connector new release. I notice for the last release we have https://mvnrepository.com/artifact/org.apache.flink/flink-connector-jdbc/3.1.1-1.16 https://mvnrepository.com/artifact/org.apache.flink/flink-connector-jdbc/3.1.1-1.17 It seems that the 2 jars above ar

Re: [DISCUSS] Kubernetes Operator 1.8.0 release planning

It's very considerate that you want to volunteer to be the release manager, but given that you have already managed one release, I would ideally like somebody else to do it. Personally, I haven't managed an operator release, although I've done it for Flink itself in the past. Nevertheless, it would

[jira] [Created] (FLINK-34407) Flaky tests causing workflow timeout

Aleksandr Pilipenko created FLINK-34407: --- Summary: Flaky tests causing workflow timeout Key: FLINK-34407 URL: https://issues.apache.org/jira/browse/FLINK-34407 Project: Flink Issue Type

Re: [DISCUSS] FLIP 411: Chaining-agnostic Operator ID generation for improved state compatibility on parallelism change

After offline discussion with @Yu Chen, I've updated the FLIP [1] to include a design that allows for compatible hasher upgrade by adding StreamGraphHasherV2 to the legacy hasher list, which is actually a revival of the idea from FLIP-5290 [2] when StreamGraphHasher

Re: [DISCUSS] Kubernetes Operator 1.8.0 release planning

If the release is postponed 1-2 more weeks, I could volunteer as the one of the release managers. Best, Rui On Wed, Feb 7, 2024 at 4:54 AM Gyula Fóra wrote: > Given the proposed timeline was a bit short / rushed I agree with Max that > we could wait 1-2 more weeks to wrap up the current outstan

[jira] [Created] (FLINK-34406) Expose more RuntimeContext functionalities in FunctionContext

yisha zhou created FLINK-34406: -- Summary: Expose more RuntimeContext functionalities in FunctionContext Key: FLINK-34406 URL: https://issues.apache.org/jira/browse/FLINK-34406 Project: Flink Is

[jira] [Created] (FLINK-34405) RightOuterJoinTaskTest#testCancelOuterJoinTaskWhileSort2 fails

Matthias Pohl created FLINK-34405: - Summary: RightOuterJoinTaskTest#testCancelOuterJoinTaskWhileSort2 fails Key: FLINK-34405 URL: https://issues.apache.org/jira/browse/FLINK-34405 Project: Flink

Re: [DISCUSS] FLIP-418: Show data skew score on Flink Dashboard

Thanks Emre for the feedback! I still think max/mean is more simple and easy to understand for users. But I don’t have a strong opinion about it. This proposal is absolutely useful for flink users! In order to ensure the value for users, would you mind if we wait for a while and check if there is

[jira] [Created] (FLINK-34404) RestoreTestBase#testRestore times out

Matthias Pohl created FLINK-34404: - Summary: RestoreTestBase#testRestore times out Key: FLINK-34404 URL: https://issues.apache.org/jira/browse/FLINK-34404 Project: Flink Issue Type: Bug