[jira] [Created] (FLINK-37162) Add Data Sources Documentation

2025-01-16 Thread Yanquan Lv (Jira)
Yanquan Lv created FLINK-37162: -- Summary: Add Data Sources Documentation Key: FLINK-37162 URL: https://issues.apache.org/jira/browse/FLINK-37162 Project: Flink Issue Type: Technical Debt

[DISCUSS] Release Flink CDC version 3.3.0

2025-01-16 Thread Hang Ruan
Hi everyone, I want to kick off the discussion for Flink CDC release v3.3.0. Flink CDC v3.3.0 [1] contains more connectors, update Flink dependency to 1.19+ and introduces many important features. I think it might be time to release v3.3.0. If there are no objections, I would like to volunteer a

Re:Re: [DISCUSSION] flink-connector-hive first release of externalized connector

2025-01-16 Thread Xuyang
Hi, Sergey. Thanks for driving this. +1 for this great work. -- Best! Xuyang At 2025-01-16 22:40:33, "Ahmed Hamdy" wrote: >Hi Sergey, > >+1 for restarting this. > >Best Regards >Ahmed Hamdy > > >On Thu, 16 Jan 2025 at 14:02, David Radley wrote: > >> Hi Sergey, >> I think this

[jira] [Created] (FLINK-37161) Cross-team verification for "Adaptive skewed join optimization for batch jobs"

2025-01-16 Thread Junrui Lee (Jira)
Junrui Lee created FLINK-37161: -- Summary: Cross-team verification for "Adaptive skewed join optimization for batch jobs" Key: FLINK-37161 URL: https://issues.apache.org/jira/browse/FLINK-37161 Project: F

[jira] [Created] (FLINK-37160) Cross-team verification for "Adaptive broadcast join for batch jobs"

2025-01-16 Thread Junrui Lee (Jira)
Junrui Lee created FLINK-37160: -- Summary: Cross-team verification for "Adaptive broadcast join for batch jobs" Key: FLINK-37160 URL: https://issues.apache.org/jira/browse/FLINK-37160 Project: Flink

[jira] [Created] (FLINK-37159) MemoryManagerConcurrentModReleaseTest.testConcurrentModificationWhileReleasing timeout in AZP

2025-01-16 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-37159: -- Summary: MemoryManagerConcurrentModReleaseTest.testConcurrentModificationWhileReleasing timeout in AZP Key: FLINK-37159 URL: https://issues.apache.org/jira/browse/FLINK-37159

[jira] [Created] (FLINK-37158) Introduce ForSt to existing ITCases

2025-01-16 Thread Yanfei Lei (Jira)
Yanfei Lei created FLINK-37158: -- Summary: Introduce ForSt to existing ITCases Key: FLINK-37158 URL: https://issues.apache.org/jira/browse/FLINK-37158 Project: Flink Issue Type: Sub-task

Re: Stale PR Action running but the labels are missing

2025-01-16 Thread Rui Fan
Hi Tom, I searched the PR list based on "This PR is being marked as stale since it has not had any activity in the last 180 days." I call it list1[1], I have added the stale labels to all missed PRs batchly from this list1[1]. And I searched the PR list by 'stale' label[2] (I call it list2), the

[jira] [Created] (FLINK-37157) Enable daily benchmark for Java 21

2025-01-16 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-37157: --- Summary: Enable daily benchmark for Java 21 Key: FLINK-37157 URL: https://issues.apache.org/jira/browse/FLINK-37157 Project: Flink Issue Type: Bug Co

[jira] [Created] (FLINK-37156) A Flink pipeline supports collecting data once and writing it to multiple sinks.

2025-01-16 Thread cheng qian (Jira)
cheng qian created FLINK-37156: -- Summary: A Flink pipeline supports collecting data once and writing it to multiple sinks. Key: FLINK-37156 URL: https://issues.apache.org/jira/browse/FLINK-37156 Project:

[DISCUSS] FLIP-505: Flink History Server Scability Improvements, Remote Data Store Fetch and Per Job Fetch

2025-01-16 Thread Allison
Hi everyone, I would like to initiate a discussion for the FLIP below, which enhances to the Flink History Server to allow greater scalability of the service. Motivation: Currently, the Flink History Server (FHS) is limited in the number of job archives it can serve based on the storage capacity

Re: Re: Re: Re: [DISCUSS] Early Fire Support for Flink SQL Interval Join

2025-01-16 Thread Venkatakrishnan Sowrirajan
Weiqing, Thanks, this is a great addition to Flink SQL. Also instead of controlling and configuring through Flink configs unlike the older window aggregation, hints seems to be a much better approach. This enables a customizable early fire behavior for individual interval joins. Couple of questio

[jira] [Created] (FLINK-37155) FLIP-505: Flink History Server Scability Improvements, Remote Data Store Fetch and Per Job Fetch

2025-01-16 Thread Allison Chang (Jira)
Allison Chang created FLINK-37155: - Summary: FLIP-505: Flink History Server Scability Improvements, Remote Data Store Fetch and Per Job Fetch Key: FLINK-37155 URL: https://issues.apache.org/jira/browse/FLINK-37155

Re: Upgrade to Netty 4.x (1.19.2, 1.20.1 releases)

2025-01-16 Thread Ferenc Csaky
Thanks for the summary Alex! +1 to keep Netty4. Best, Ferenc On Thursday, January 16th, 2025 at 17:02, Tom Cooper wrote: > > > + 1 on keeping the upgraded Pekko for 1.20.1. > > From what I could see, the only issues occurred in a end-to-end test that set > the available memory to 7MB. >

Re: Upgrade to Netty 4.x (1.19.2, 1.20.1 releases)

2025-01-16 Thread Tom Cooper
+ 1 on keeping the upgraded Pekko for 1.20.1. >From what I could see, the only issues occurred in a end-to-end test that set >the available memory to 7MB. As that is unlikely to be a situation seen in the wild, I think the CVE fixes alone are worth it. Thanks, Tom Cooper @tomncooper | tomcoop

Re: Upgrade to Netty 4.x (1.19.2, 1.20.1 releases)

2025-01-16 Thread Gyula Fóra
Hello, +1 for moving to the new version and accepting the minimally increased memory consumption. This is extremely unlikely to cause any issues in practice that cannot be fixed almost instantly on the user side. This is the right and future proof thing to do. Cheers, Gyula On Thu, Jan 16, 2025

Re: Upgrade to Netty 4.x (1.19.2, 1.20.1 releases)

2025-01-16 Thread Alexander Fedulov
cc Norman On Thu, 16 Jan 2025 at 16:12, Alexander Fedulov wrote: > Hi all, > > We have one remaining blocker for the 1.19.2 and 1.20.1 releases, namely > the issue associated with ticket FLINK-36510: *"Upgrade Pekko from 1.0.1 > to 1.1.2"* [1]. Here is the context: > >- The flink-rpc module

Upgrade to Netty 4.x (1.19.2, 1.20.1 releases)

2025-01-16 Thread Alexander Fedulov
Hi all, We have one remaining blocker for the 1.19.2 and 1.20.1 releases, namely the issue associated with ticket FLINK-36510: *"Upgrade Pekko from 1.0.1 to 1.1.2"* [1]. Here is the context: - The flink-rpc module is currently based on Pekko 1.0.1, which bundles Netty version 3.10.6. Netty

Re: [DISCUSSION] flink-connector-hive first release of externalized connector

2025-01-16 Thread Ahmed Hamdy
Hi Sergey, +1 for restarting this. Best Regards Ahmed Hamdy On Thu, 16 Jan 2025 at 14:02, David Radley wrote: > Hi Sergey, > I think this makes a lot of sense to get this cleared up soon. > I notice that there is a HiveCatalog referred to in the Python in core > Flink. I suspect this relates

FW: Community Health Initiative (CHI) 9th of January minutes

2025-01-16 Thread David Radley
Hi We ran the 7th Community Health Initiative (CHI) on the 16th of January details https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=337678600 The high level summary: * We have identified 10 PRs that should be merged – please could a committer merge these. * We have activa

Re: [DISCUSSION] flink-connector-hive first release of externalized connector

2025-01-16 Thread David Radley
Hi Sergey, I think this makes a lot of sense to get this cleared up soon. I notice that there is a HiveCatalog referred to in the Python in core Flink. I suspect this relates to the HiveCatalog class in core Flink. Do we need to remove the Hive part of Python as part of this , or potentially ch

[jira] [Created] (FLINK-37154) Support reading binary data from Kafka

2025-01-16 Thread Dian Fu (Jira)
Dian Fu created FLINK-37154: --- Summary: Support reading binary data from Kafka Key: FLINK-37154 URL: https://issues.apache.org/jira/browse/FLINK-37154 Project: Flink Issue Type: Improvement

Re: Stale PR Action running but the labels are missing

2025-01-16 Thread Tom Cooper
Hi Rui, After further discussion, on the CHI call this morning, we decided it is best to just leave the action and its current state (with the updated message). The GH Actions have their own state cache, so the action doesn't use the labels to tell what it marked as stale in the past. To help

[jira] [Created] (FLINK-37153) Monitor late event count in temporal join

2025-01-16 Thread Jira
Grzegorz Kołakowski created FLINK-37153: --- Summary: Monitor late event count in temporal join Key: FLINK-37153 URL: https://issues.apache.org/jira/browse/FLINK-37153 Project: Flink Issue

[jira] [Created] (FLINK-37152) Update Flink version to 1.20 in Flink CDC

2025-01-16 Thread Ferenc Csaky (Jira)
Ferenc Csaky created FLINK-37152: Summary: Update Flink version to 1.20 in Flink CDC Key: FLINK-37152 URL: https://issues.apache.org/jira/browse/FLINK-37152 Project: Flink Issue Type: Improve

[jira] [Created] (FLINK-37151) Remove reduandant testing dependencies of Flink UI

2025-01-16 Thread Ao Yuchen (Jira)
Ao Yuchen created FLINK-37151: - Summary: Remove reduandant testing dependencies of Flink UI Key: FLINK-37151 URL: https://issues.apache.org/jira/browse/FLINK-37151 Project: Flink Issue Type: Impr

[jira] [Created] (FLINK-37150) Adjust the documentation of the configuration file to remove the support for the old configuration file."

2025-01-16 Thread Junrui Lee (Jira)
Junrui Lee created FLINK-37150: -- Summary: Adjust the documentation of the configuration file to remove the support for the old configuration file." Key: FLINK-37150 URL: https://issues.apache.org/jira/browse/FLINK-37

[jira] [Created] (FLINK-37149) Add Adaptive Batch Execution Documentation

2025-01-16 Thread Junrui Lee (Jira)
Junrui Lee created FLINK-37149: -- Summary: Add Adaptive Batch Execution Documentation Key: FLINK-37149 URL: https://issues.apache.org/jira/browse/FLINK-37149 Project: Flink Issue Type: Sub-task