Re: [PR] [FLINK-33874][runtime] Support resource request wait mechanism at DefaultDeclarativeSlotPool side for Default Scheduler [flink]

2024-07-25 Thread via GitHub
RocMarshal commented on code in PR #25113: URL: https://github.com/apache/flink/pull/25113#discussion_r1689451505 ## flink-runtime/src/main/java/org/apache/flink/runtime/jobmaster/slotpool/DeclarativeSlotPoolService.java: ## @@ -81,18 +84,26 @@ public DeclarativeSlotPoolService(

Re: [PR] [FLINK-35864][Table SQL / API] Add CONV function [flink]

2024-07-25 Thread via GitHub
superdiaodiao commented on code in PR #25109: URL: https://github.com/apache/flink/pull/25109#discussion_r1690918688 ## flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/MathFunctionsITCase.java: ## @@ -141,4 +178,181 @@ Stream getTestSetSpec

Re: [PR] [FLINK-35864][Table SQL / API] Add CONV function [flink]

2024-07-25 Thread via GitHub
superdiaodiao commented on code in PR #25109: URL: https://github.com/apache/flink/pull/25109#discussion_r1690918688 ## flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/MathFunctionsITCase.java: ## @@ -141,4 +178,181 @@ Stream getTestSetSpec

Re: [PR] [FLINK-35864][Table SQL / API] Add CONV function [flink]

2024-07-25 Thread via GitHub
superdiaodiao commented on code in PR #25109: URL: https://github.com/apache/flink/pull/25109#discussion_r1690918688 ## flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/MathFunctionsITCase.java: ## @@ -141,4 +178,181 @@ Stream getTestSetSpec

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-25 Thread via GitHub
mateczagany commented on code in PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#discussion_r1690995181 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/config/FlinkConfigManager.java: ## @@ -72,21 +74,26 @@ public class

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-25 Thread via GitHub
mateczagany commented on code in PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#discussion_r1690999244 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/utils/FlinkStateSnapshotUtils.java: ## @@ -0,0 +1,395 @@ +/* + * Li

Re: [PR] [FLINK-35864][Table SQL / API] Add CONV function [flink]

2024-07-25 Thread via GitHub
dylanhz commented on code in PR #25109: URL: https://github.com/apache/flink/pull/25109#discussion_r1691012678 ## flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/MathFunctionsITCase.java: ## @@ -141,4 +178,181 @@ Stream getTestSetSpecs() {

Re: [PR] [BP-1.20][FLINK-35095][test] Fix unstable tests in `ExecutionEnvironmentImplTest` [flink]

2024-07-25 Thread via GitHub
reswqa commented on PR #25111: URL: https://github.com/apache/flink/pull/25111#issuecomment-2249702180 Already reviewed in https://github.com/apache/flink/pull/25085. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] [BP-1.20][FLINK-35095][test] Fix unstable tests in `ExecutionEnvironmentImplTest` [flink]

2024-07-25 Thread via GitHub
reswqa merged PR #25111: URL: https://github.com/apache/flink/pull/25111 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

Re: [PR] [FLINK-33874][runtime] Support resource request wait mechanism at DefaultDeclarativeSlotPool side for Default Scheduler [flink]

2024-07-25 Thread via GitHub
RocMarshal commented on code in PR #25113: URL: https://github.com/apache/flink/pull/25113#discussion_r1690890786 ## flink-runtime/src/test/java/org/apache/flink/runtime/jobmaster/slotpool/BlocklistDeclarativeSlotPoolTest.java: ## @@ -154,7 +163,7 @@ private void testRegisterSlo

Re: [PR] [FLINK-33874][runtime] Support resource request wait mechanism at DefaultDeclarativeSlotPool side for Default Scheduler [flink]

2024-07-25 Thread via GitHub
1996fanrui commented on code in PR #25113: URL: https://github.com/apache/flink/pull/25113#discussion_r1690930831 ## flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/ResourceManagerGateway.java: ## @@ -88,7 +88,22 @@ CompletableFuture registerJobMaster(

Re: [PR] [FLINK-33874][runtime] Support resource request wait mechanism at DefaultDeclarativeSlotPool side for Default Scheduler [flink]

2024-07-25 Thread via GitHub
1996fanrui commented on code in PR #25113: URL: https://github.com/apache/flink/pull/25113#discussion_r1690930831 ## flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/ResourceManagerGateway.java: ## @@ -88,7 +88,22 @@ CompletableFuture registerJobMaster(

Re: [PR] [hotfix][test] Add CDC 3.1.1 release to migration test versions [flink-cdc]

2024-07-25 Thread via GitHub
leonardBang merged PR #3426: URL: https://github.com/apache/flink-cdc/pull/3426 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [FLINK-35864][Table SQL / API] Add CONV function [flink]

2024-07-25 Thread via GitHub
superdiaodiao commented on code in PR #25109: URL: https://github.com/apache/flink/pull/25109#discussion_r1691055255 ## flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/MathFunctionsITCase.java: ## @@ -141,4 +178,181 @@ Stream getTestSetSpec

[jira] [Created] (FLINK-35895) flink parquet vectorized reader should use readNextFilteredRowGroup() instead of readNextRowGroup()

2024-07-25 Thread Kai Chen (Jira)
Kai Chen created FLINK-35895: Summary: flink parquet vectorized reader should use readNextFilteredRowGroup() instead of readNextRowGroup() Key: FLINK-35895 URL: https://issues.apache.org/jira/browse/FLINK-35895

[jira] [Updated] (FLINK-35895) flink parquet vectorized reader should use readNextFilteredRowGroup() instead of readNextRowGroup()

2024-07-25 Thread Kai Chen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Chen updated FLINK-35895: - Description: reader.readNextRowGroup() should be changed to reader.readNextFilteredRowGroup(); https://

Re: [PR] [FLINK-35847][release] Add release note for version 1.20 [flink]

2024-07-25 Thread via GitHub
superdiaodiao commented on code in PR #25091: URL: https://github.com/apache/flink/pull/25091#discussion_r1691112012 ## docs/content.zh/release-notes/flink-1.20.md: ## @@ -0,0 +1,434 @@ +--- +title: "Release Notes - Flink 1.20" +--- + + +# Release notes - Flink 1.20 + +These rel

Re: [PR] [FLINK-33386][runtime] Support tasks balancing at slot level for Default Scheduler [flink]

2024-07-25 Thread via GitHub
RocMarshal commented on code in PR #23635: URL: https://github.com/apache/flink/pull/23635#discussion_r1691113488 ## flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/LocalInputPreferredSlotSharingStrategyTest.java: ## @@ -69,46 +69,29 @@ class LocalInputPreferredSl

Re: [PR] [FLINK-35847][release] Add release note for version 1.20 [flink]

2024-07-25 Thread via GitHub
superdiaodiao commented on code in PR #25091: URL: https://github.com/apache/flink/pull/25091#discussion_r1691112012 ## docs/content.zh/release-notes/flink-1.20.md: ## @@ -0,0 +1,434 @@ +--- +title: "Release Notes - Flink 1.20" +--- + + +# Release notes - Flink 1.20 + +These rel

Re: [PR] [FLINK-33874][runtime] Support resource request wait mechanism at DefaultDeclarativeSlotPool side for Default Scheduler [flink]

2024-07-25 Thread via GitHub
RocMarshal commented on code in PR #25113: URL: https://github.com/apache/flink/pull/25113#discussion_r1691120384 ## flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/ResourceManagerGateway.java: ## @@ -88,7 +88,22 @@ CompletableFuture registerJobMaster(

[jira] [Commented] (FLINK-35887) Null Pointer Exception in TypeExtractor.isRecord when trying to provide type info for interface

2024-07-25 Thread Jacob Jona Fahlenkamp (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868615#comment-17868615 ] Jacob Jona Fahlenkamp commented on FLINK-35887: --- [~dylanhz] Correct, the t

Re: [PR] [FLINK-35016] catalog changes for model resource [flink]

2024-07-25 Thread via GitHub
twalthr commented on code in PR #25036: URL: https://github.com/apache/flink/pull/25036#discussion_r1690005374 ## flink-table/flink-table-common/src/main/java/org/apache/flink/table/catalog/Catalog.java: ## @@ -790,4 +792,126 @@ void alterPartitionColumnStatistics(

[jira] [Updated] (FLINK-35894) Add Elasticsearch Sink Connector for Flink CDC Pipeline

2024-07-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-35894: --- Labels: pull-request-available (was: ) > Add Elasticsearch Sink Connector for Flink CDC Pip

Re: [PR] [FLINK-35391][cdc-connector][paimon] Bump dependency of Paimon Pipeline connector to 0.8.0 [flink-cdc]

2024-07-25 Thread via GitHub
beryllw commented on PR #3335: URL: https://github.com/apache/flink-cdc/pull/3335#issuecomment-2249967829 @leonardBang It seems that the failure of unit ci has nothing to do with Paimon's changes. -- This is an automated message from the Apache Git Service. To respond to the message, plea

[jira] [Created] (FLINK-35896) eventHandler in RescaleApiScalingRealizer handle the event of scaling failure

2024-07-25 Thread yuanfenghu (Jira)
yuanfenghu created FLINK-35896: -- Summary: eventHandler in RescaleApiScalingRealizer handle the event of scaling failure Key: FLINK-35896 URL: https://issues.apache.org/jira/browse/FLINK-35896 Project: F

[jira] [Commented] (FLINK-35896) eventHandler in RescaleApiScalingRealizer handle the event of scaling failure

2024-07-25 Thread yuanfenghu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868629#comment-17868629 ] yuanfenghu commented on FLINK-35896: [~fanrui]  Can you give me some advice on this

[jira] [Comment Edited] (FLINK-35896) eventHandler in RescaleApiScalingRealizer handle the event of scaling failure

2024-07-25 Thread yuanfenghu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868631#comment-17868631 ] yuanfenghu edited comment on FLINK-35896 at 7/25/24 10:23 AM:

[jira] [Commented] (FLINK-35896) eventHandler in RescaleApiScalingRealizer handle the event of scaling failure

2024-07-25 Thread yuanfenghu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868631#comment-17868631 ] yuanfenghu commented on FLINK-35896: When an exception occurs in ScalingRealizer.rea

[jira] [Commented] (FLINK-35896) eventHandler in RescaleApiScalingRealizer handle the event of scaling failure

2024-07-25 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868635#comment-17868635 ] Rui Fan commented on FLINK-35896: - RescaleApiScalingRealizer#realizeParallelismOverrides

[jira] [Commented] (FLINK-35896) eventHandler in RescaleApiScalingRealizer handle the event of scaling failure

2024-07-25 Thread yuanfenghu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868639#comment-17868639 ] yuanfenghu commented on FLINK-35896: > RescaleApiScalingRealizer#realizeParallelismO

[jira] [Assigned] (FLINK-35894) Add Elasticsearch Sink Connector for Flink CDC Pipeline

2024-07-25 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-35894: -- Assignee: wuzexian > Add Elasticsearch Sink Connector for Flink CDC Pipeline > --

[jira] [Updated] (FLINK-35868) Bump Mongo driver version to support Mongo 7.0+

2024-07-25 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-35868: --- Fix Version/s: cdc-3.2.0 > Bump Mongo driver version to support Mongo 7.0+ > ---

Re: [PR] [FLINK-35868][cdc-connector][mongodb] Bump dependency version to support MongoDB 7.0 [flink-cdc]

2024-07-25 Thread via GitHub
leonardBang merged PR #3489: URL: https://github.com/apache/flink-cdc/pull/3489 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

[jira] [Updated] (FLINK-35868) Bump Mongo driver version to support Mongo 7.0+

2024-07-25 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-35868: --- Affects Version/s: cdc-3.1.1 > Bump Mongo driver version to support Mongo 7.0+ > ---

[jira] [Resolved] (FLINK-35868) Bump Mongo driver version to support Mongo 7.0+

2024-07-25 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-35868. Resolution: Implemented via master: 241eb03a74c521932ee5a4ba92a4ac479f9123a4 > Bump Mongo driver

[jira] [Commented] (FLINK-33555) LocalRecoveryITCase.testRecoverLocallyFromProcessCrashWithWorkingDirectory:

2024-07-25 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868648#comment-17868648 ] Weijie Guo commented on FLINK-33555: https://dev.azure.com/apache-flink/apache-flink

[jira] [Commented] (FLINK-35810) AsyncWaitOperatorTest.testProcessingTimeRepeatedCompleteUnorderedWithRetry fails

2024-07-25 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868649#comment-17868649 ] Weijie Guo commented on FLINK-35810: I saw an OOM before this test failure. {code:

[jira] [Commented] (FLINK-35885) proctime aggregate window triggered by watermark

2024-07-25 Thread xuyang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868658#comment-17868658 ] xuyang commented on FLINK-35885: The bug is caused by the advancement of the watermark a

[jira] [Commented] (FLINK-35885) proctime aggregate window triggered by watermark

2024-07-25 Thread xuyang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868659#comment-17868659 ] xuyang commented on FLINK-35885: I think this bug needs to be fixed although it only aff

Re: [PR] [FLINK-35894] Add Elasticsearch Sink Connector for Flink CDC Pipeline [flink-cdc]

2024-07-25 Thread via GitHub
lvyanquan commented on code in PR #3495: URL: https://github.com/apache/flink-cdc/pull/3495#discussion_r1691213209 ## flink-cdc-connect/flink-cdc-pipeline-connectors/flink-cdc-pipeline-connector-elasticsearch/pom.xml: ## @@ -0,0 +1,211 @@ +http://maven.apache.org/POM/4.0.0"; +

Re: [PR] [FLINK-35894] Add Elasticsearch Sink Connector for Flink CDC Pipeline [flink-cdc]

2024-07-25 Thread via GitHub
lvyanquan commented on PR #3495: URL: https://github.com/apache/flink-cdc/pull/3495#issuecomment-2250180097 Thanks @proletarians for this contribution, left some comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[jira] [Comment Edited] (FLINK-35885) proctime aggregate window triggered by watermark

2024-07-25 Thread xuyang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868659#comment-17868659 ] xuyang edited comment on FLINK-35885 at 7/25/24 12:24 PM: -- I th

Re: [PR] [FLINK-35282][FLINK-35520] PyFlink Support for Apache Beam > 2.49 [flink]

2024-07-25 Thread via GitHub
xaniasd commented on PR #24908: URL: https://github.com/apache/flink/pull/24908#issuecomment-2250200976 hi there @snuyanzin @hlteoh37, are you actively working on this PR? Is there any way I could help with this? -- This is an automated message from the Apache Git Service. To respond to t

[jira] [Updated] (FLINK-35885) proctime aggregate window triggered by watermark

2024-07-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-35885: --- Labels: pull-request-available (was: ) > proctime aggregate window triggered by watermark >

[PR] [FLINK-35885][table] Ignore advancing window processor by watermark in window agg based on proctime [flink]

2024-07-25 Thread via GitHub
xuyangzhong opened a new pull request, #25119: URL: https://github.com/apache/flink/pull/25119 ## What is the purpose of the change This pr tries to ignore the advancement of the window processor caused by watermark in proctime window agg. ## Brief change log - *Ignore

Re: [PR] [FLINK-35885][table] Ignore advancing window processor by watermark in window agg based on proctime [flink]

2024-07-25 Thread via GitHub
flinkbot commented on PR #25119: URL: https://github.com/apache/flink/pull/25119#issuecomment-2250260386 ## CI report: * f529368b2ff98c7a50d5e790c9b9ca879ca200a8 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Created] (FLINK-35897) Some checkpoint files and localState files can't be cleanUp when checkpoint is aborted

2024-07-25 Thread Jinzhong Li (Jira)
Jinzhong Li created FLINK-35897: --- Summary: Some checkpoint files and localState files can't be cleanUp when checkpoint is aborted Key: FLINK-35897 URL: https://issues.apache.org/jira/browse/FLINK-35897

[jira] [Updated] (FLINK-35897) Some checkpoint files and localState files can't be cleanUp when checkpoint is aborted

2024-07-25 Thread Jinzhong Li (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinzhong Li updated FLINK-35897: Description: h2. Problem When the job checkpoint is canceled ([asyncsnapshotcallable.java/#L129|

[jira] [Updated] (FLINK-35897) Some checkpoint files and localState files can't be cleanUp when checkpoint is aborted

2024-07-25 Thread Jinzhong Li (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinzhong Li updated FLINK-35897: Description: h2. Problem When the job checkpoint is canceled ([asyncsnapshotcallable.java/#L129|

[jira] [Updated] (FLINK-35897) Some checkpoint files and localState files can't be cleanUp when checkpoint is aborted

2024-07-25 Thread Jinzhong Li (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinzhong Li updated FLINK-35897: Description: h2. Problem When the job checkpoint is canceled ([asyncsnapshotcallable.java/#L129|

[jira] [Created] (FLINK-35898) When running Flink on Yarn, if the log file path contains a space, the startup will fail.

2024-07-25 Thread yiheng tang (Jira)
yiheng tang created FLINK-35898: --- Summary: When running Flink on Yarn, if the log file path contains a space, the startup will fail. Key: FLINK-35898 URL: https://issues.apache.org/jira/browse/FLINK-35898

[jira] [Updated] (FLINK-35898) When running Flink on Yarn, if the log file path contains a space, the startup will fail.

2024-07-25 Thread yiheng tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yiheng tang updated FLINK-35898: Attachment: image-2024-07-25-21-51-50-072.png > When running Flink on Yarn, if the log file path c

[jira] [Updated] (FLINK-35898) When running Flink on Yarn, if the log file path contains a space, the startup will fail.

2024-07-25 Thread yiheng tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yiheng tang updated FLINK-35898: Attachment: image-2024-07-25-21-52-26-973.png > When running Flink on Yarn, if the log file path c

[jira] [Updated] (FLINK-35898) When running Flink on Yarn, if the log file path contains a space, the startup will fail.

2024-07-25 Thread yiheng tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yiheng tang updated FLINK-35898: Description: We use Yarn to launch Flink jobs and try to add custom parameters to configure the l

[PR] [FLINK-35897][Checkpoint] Cleanup completed resource when checkpoint is canceled [flink]

2024-07-25 Thread via GitHub
ljz2051 opened a new pull request, #25120: URL: https://github.com/apache/flink/pull/25120 ## What is the purpose of the change This pull request cleanup completed checkpoint resource when the job checkpoint is canceled. ## Brief change log - When the asynchr

[jira] [Updated] (FLINK-35897) Some checkpoint files and localState files can't be cleanUp when checkpoint is aborted

2024-07-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-35897: --- Labels: pull-request-available (was: ) > Some checkpoint files and localState files can't b

Re: [PR] [FLINK-35897][Checkpoint] Cleanup completed resource when checkpoint is canceled [flink]

2024-07-25 Thread via GitHub
flinkbot commented on PR #25120: URL: https://github.com/apache/flink/pull/25120#issuecomment-2250449661 ## CI report: * 5b0ca74d3951873a800d5b3ef0d23736f6fab755 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Created] (FLINK-35899) Accumulated TumblingProcessingTimeWindows in the state for a few days

2024-07-25 Thread Sergey Anokhovskiy (Jira)
Sergey Anokhovskiy created FLINK-35899: -- Summary: Accumulated TumblingProcessingTimeWindows in the state for a few days Key: FLINK-35899 URL: https://issues.apache.org/jira/browse/FLINK-35899 Pro

[jira] [Updated] (FLINK-35899) Accumulated TumblingProcessingTimeWindows in the state for a few days

2024-07-25 Thread Sergey Anokhovskiy (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Anokhovskiy updated FLINK-35899: --- Description: One of the sub-task out of 40 of the TumblingProcessingTimeWindows oper

[jira] [Updated] (FLINK-35899) Accumulated TumblingProcessingTimeWindows in the state for a few days

2024-07-25 Thread Sergey Anokhovskiy (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Anokhovskiy updated FLINK-35899: --- Description: One of the sub-task out of 40 of the TumblingProcessingTimeWindows oper

[jira] [Commented] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-25 Thread Trystan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868711#comment-17868711 ] Trystan commented on FLINK-35285: - Thank you both for explaining the nuances of this - i

[jira] [Updated] (FLINK-35899) Accumulated TumblingProcessingTimeWindows in the state for a few days

2024-07-25 Thread Sergey Anokhovskiy (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Anokhovskiy updated FLINK-35899: --- Attachment: dedup_window_intializing_slowly.png dedup_window_processe

[jira] [Updated] (FLINK-35899) Accumulated TumblingProcessingTimeWindows in the state for a few days

2024-07-25 Thread Sergey Anokhovskiy (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Anokhovskiy updated FLINK-35899: --- Attachment: sub_task_state_size.txt > Accumulated TumblingProcessingTimeWindows in t

[jira] [Updated] (FLINK-35899) Accumulated TumblingProcessingTimeWindows in the state for a few days

2024-07-25 Thread Sergey Anokhovskiy (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Anokhovskiy updated FLINK-35899: --- Description: One of the sub-task out of 40 of the TumblingProcessingTimeWindows oper

[jira] [Updated] (FLINK-35899) Accumulated TumblingProcessingTimeWindows in the state for a few days

2024-07-25 Thread Sergey Anokhovskiy (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Anokhovskiy updated FLINK-35899: --- Description: One of the sub-task out of 40 of the TumblingProcessingTimeWindows oper

[jira] [Updated] (FLINK-35899) Accumulated TumblingProcessingTimeWindows in the state for a few days

2024-07-25 Thread Sergey Anokhovskiy (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Anokhovskiy updated FLINK-35899: --- Attachment: ids_destribution.txt > Accumulated TumblingProcessingTimeWindows in the

[jira] [Updated] (FLINK-35899) Accumulated TumblingProcessingTimeWindows in the state for a few days

2024-07-25 Thread Sergey Anokhovskiy (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Anokhovskiy updated FLINK-35899: --- Attachment: timers.txt > Accumulated TumblingProcessingTimeWindows in the state for

[jira] [Updated] (FLINK-35899) Accumulated TumblingProcessingTimeWindows in the state for a few days

2024-07-25 Thread Sergey Anokhovskiy (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Anokhovskiy updated FLINK-35899: --- Attachment: taskmanager_exception.txt > Accumulated TumblingProcessingTimeWindows in

[jira] [Updated] (FLINK-35899) Accumulated TumblingProcessingTimeWindows in the state for a few days

2024-07-25 Thread Sergey Anokhovskiy (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Anokhovskiy updated FLINK-35899: --- Description: One of the sub-task out of 40 of the TumblingProcessingTimeWindows oper

[jira] [Updated] (FLINK-35899) Accumulated TumblingProcessingTimeWindows in the state for a few days

2024-07-25 Thread Sergey Anokhovskiy (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Anokhovskiy updated FLINK-35899: --- Description: One of the sub-task out of 40 of the TumblingProcessingTimeWindows oper

[jira] [Comment Edited] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-25 Thread Trystan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868711#comment-17868711 ] Trystan edited comment on FLINK-35285 at 7/25/24 3:19 PM: -- Than

[jira] [Commented] (FLINK-35898) When running Flink on Yarn, if the log file path contains a space, the startup will fail.

2024-07-25 Thread Ferenc Csaky (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868717#comment-17868717 ] Ferenc Csaky commented on FLINK-35898: -- I am not sure I understand how the passed {

[jira] [Commented] (FLINK-35672) testPreAggregatedSlidingTimeWindow failed due to due to checkpoint expired before completing

2024-07-25 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868720#comment-17868720 ] Matthias Pohl commented on FLINK-35672: --- The actual error seems to be the followin

[jira] [Updated] (FLINK-35672) testPreAggregatedSlidingTimeWindow failed due to due to checkpoint expired before completing

2024-07-25 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-35672: -- Attachment: FLINK-35672.log > testPreAggregatedSlidingTimeWindow failed due to due to checkpoi

[jira] [Comment Edited] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-25 Thread Trystan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868711#comment-17868711 ] Trystan edited comment on FLINK-35285 at 7/25/24 3:33 PM: -- Than

[jira] [Commented] (FLINK-35499) EventTimeWindowCheckpointingITCase times out due to Checkpoint expired before completing

2024-07-25 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868721#comment-17868721 ] Matthias Pohl commented on FLINK-35499: --- Looks like FLINK-35672 and FLINK-35499 ar

[jira] [Updated] (FLINK-35499) EventTimeWindowCheckpointingITCase times out due to Checkpoint expired before completing

2024-07-25 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-35499: -- Attachment: FLINK-35499.AdaptiveScheduler.log > EventTimeWindowCheckpointingITCase times out d

[jira] [Closed] (FLINK-35672) testPreAggregatedSlidingTimeWindow failed due to due to checkpoint expired before completing

2024-07-25 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl closed FLINK-35672. - Resolution: Duplicate I'm closing this one as a duplicate of FLINK-35499 > testPreAggregatedSli

[jira] [Commented] (FLINK-35499) EventTimeWindowCheckpointingITCase times out due to Checkpoint expired before completing

2024-07-25 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868723#comment-17868723 ] Matthias Pohl commented on FLINK-35499: --- Test failure copied over from FLINK-35672

[jira] [Updated] (FLINK-35499) EventTimeWindowCheckpointingITCase times out due to Checkpoint expired before completing

2024-07-25 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-35499: -- Component/s: Runtime / Checkpointing > EventTimeWindowCheckpointingITCase times out due to Che

[jira] [Commented] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-25 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868726#comment-17868726 ] Gyula Fora commented on FLINK-35285: [~trystan] let meg give you a somewhat artifica

[jira] [Created] (FLINK-35900) PyFlink: update version of Apache Beam dependency

2024-07-25 Thread Lydia L (Jira)
Lydia L created FLINK-35900: --- Summary: PyFlink: update version of Apache Beam dependency Key: FLINK-35900 URL: https://issues.apache.org/jira/browse/FLINK-35900 Project: Flink Issue Type: Bug A

[jira] [Updated] (FLINK-35900) PyFlink: update version of Apache Beam dependency

2024-07-25 Thread Lydia L (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lydia L updated FLINK-35900: Priority: Blocker (was: Critical) > PyFlink: update version of Apache Beam dependency > -

[jira] [Created] (FLINK-35901) Setting web.submit.enable to false prevents FlinkSessionJobs from working

2024-07-25 Thread Ralph Blaise (Jira)
Ralph Blaise created FLINK-35901: Summary: Setting web.submit.enable to false prevents FlinkSessionJobs from working Key: FLINK-35901 URL: https://issues.apache.org/jira/browse/FLINK-35901 Project: Fl

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-25 Thread via GitHub
anupamaggarwal commented on code in PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#discussion_r1691872393 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/reconciler/snapshot/StateSnapshotReconciler.java: ## @@ -0,0 +1,

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-25 Thread via GitHub
anupamaggarwal commented on code in PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#discussion_r1691872393 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/reconciler/snapshot/StateSnapshotReconciler.java: ## @@ -0,0 +1,

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-25 Thread via GitHub
anupamaggarwal commented on code in PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#discussion_r1691868012 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/utils/FlinkStateSnapshotUtils.java: ## @@ -0,0 +1,382 @@ +/* + *

[jira] [Commented] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-25 Thread Maximilian Michels (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868752#comment-17868752 ] Maximilian Michels commented on FLINK-35285: To add, key group alignment ens

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-25 Thread via GitHub
mateczagany commented on code in PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#discussion_r1691933742 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/reconciler/snapshot/StateSnapshotReconciler.java: ## @@ -0,0 +1,230

Re: [PR] [FLINK-35305]Amazon SQS Sink Connector [flink-connector-aws]

2024-07-25 Thread via GitHub
19priyadhingra commented on PR #141: URL: https://github.com/apache/flink-connector-aws/pull/141#issuecomment-2251149507 > @19priyadhingra The tests seems to be failing. Can we please take a look? > > Also - it would be good if we squash the commits! 1. Hi Hong, I tried to deep

Re: [PR] [FLINK-30481][FLIP-277] GlueCatalog Implementation [flink-connector-aws]

2024-07-25 Thread via GitHub
Samrat002 commented on code in PR #47: URL: https://github.com/apache/flink-connector-aws/pull/47#discussion_r1691963997 ## flink-catalog-aws/flink-catalog-aws-glue/src/main/java/org/apache/flink/table/catalog/glue/util/GlueUtils.java: ## @@ -0,0 +1,384 @@ +/* + * Licensed to t

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-25 Thread via GitHub
mateczagany commented on code in PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#discussion_r1691962117 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/utils/FlinkStateSnapshotUtils.java: ## @@ -0,0 +1,382 @@ +/* + * Li

Re: [PR] [FLINK-35736][tests] Add migration test scripts & CI workflows [flink-cdc]

2024-07-25 Thread via GitHub
morazow commented on code in PR #3447: URL: https://github.com/apache/flink-cdc/pull/3447#discussion_r1691956059 ## tools/mig-test/README.md: ## @@ -0,0 +1,36 @@ +# Flink CDC Migration Test Utilities + +## Pipeline Jobs +### Preparation + +1. Install Ruby (macOS has embedded it

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-25 Thread via GitHub
mateczagany commented on code in PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#discussion_r1691974263 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/service/AbstractFlinkService.java: ## @@ -716,6 +695,48 @@ public C

Re: [PR] [FLINK-30481][FLIP-277] GlueCatalog Implementation [flink-connector-aws]

2024-07-25 Thread via GitHub
Samrat002 commented on PR #47: URL: https://github.com/apache/flink-connector-aws/pull/47#issuecomment-2251178966 Thank you @foxus for the review . this PR was opened long ago. Let me try to address all the comments and test it again e2e . this PR will be ready for review by next 2

[jira] [Commented] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-25 Thread Trystan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868783#comment-17868783 ] Trystan commented on FLINK-35285: - Thank you both again, that's the aha-moment I needed!

[jira] [Commented] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-25 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868784#comment-17868784 ] Gyula Fora commented on FLINK-35285: What we currently don't have is any measurement

[jira] [Assigned] (FLINK-35896) eventHandler in RescaleApiScalingRealizer handle the event of scaling failure

2024-07-25 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Fan reassigned FLINK-35896: --- Assignee: yuanfenghu > eventHandler in RescaleApiScalingRealizer handle the event of scaling failu

[jira] [Resolved] (FLINK-33874) Support resource request wait mechanism at DefaultDeclarativeSlotPool side for Default Scheduler

2024-07-25 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Fan resolved FLINK-33874. - Fix Version/s: 2.0.0 Resolution: Fixed > Support resource request wait mechanism at DefaultDeclar

Re: [PR] [FLINK-33874][runtime] Support resource request wait mechanism at DefaultDeclarativeSlotPool side for Default Scheduler [flink]

2024-07-25 Thread via GitHub
1996fanrui merged PR #25113: URL: https://github.com/apache/flink/pull/25113 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.ap

  1   2   >