[jira] [Comment Edited] (FLINK-35887) Null Pointer Exception in TypeExtractor.isRecord when trying to provide type info for interface

2024-07-23 Thread Dylan He (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868246#comment-17868246 ] Dylan He edited comment on FLINK-35887 at 7/24/24 5:36 AM: --- I

Re: [PR] [FLINK-35868][cdc-connector][mongodb] Bump dependency version to support MongoDB 7.0 [flink-cdc]

2024-07-23 Thread via GitHub
yuxiqian commented on code in PR #3489: URL: https://github.com/apache/flink-cdc/pull/3489#discussion_r1689156469 ## docs/content.zh/docs/connectors/flink-sources/overview.md: ## @@ -37,7 +37,7 @@ You can also read [tutorials]({{< ref "docs/connectors/flink-sources/tutorials/b

Re: [PR] [FLINK-35868][cdc-connector][mongodb] Bump dependency version to support MongoDB 7.0 [flink-cdc]

2024-07-23 Thread via GitHub
Jiabao-Sun commented on code in PR #3489: URL: https://github.com/apache/flink-cdc/pull/3489#discussion_r1689134338 ## docs/content.zh/docs/connectors/flink-sources/overview.md: ## @@ -37,7 +37,7 @@ You can also read [tutorials]({{< ref "docs/connectors/flink-sources/tutorials/

Re: [PR] [FLINK-35857] fix redeploy failed deployment without latest checkpoint [flink-kubernetes-operator]

2024-07-23 Thread via GitHub
gyfora commented on code in PR #855: URL: https://github.com/apache/flink-kubernetes-operator/pull/855#discussion_r1689124506 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/reconciler/deployment/AbstractJobReconciler.java: ## @@ -313,9 +313,19 @

[jira] [Commented] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-23 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868252#comment-17868252 ] Gyula Fora commented on FLINK-35285: Looping in [~mxm] as he may have input here too

[jira] [Commented] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-23 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868251#comment-17868251 ] Gyula Fora commented on FLINK-35285: "always act" would only make sense if we disabl

[jira] [Commented] (FLINK-35887) Null Pointer Exception in TypeExtractor.isRecord when trying to provide type info for interface

2024-07-23 Thread Dylan He (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868246#comment-17868246 ] Dylan He commented on FLINK-35887: -- I have solved the problem, but what is the practica

Re: [PR] [FLINK-35835][test] Make file-merging test tolerate the scenario that source does not produce any record [flink]

2024-07-23 Thread via GitHub
flinkbot commented on PR #25116: URL: https://github.com/apache/flink/pull/25116#issuecomment-2246809131 ## CI report: * a2b0a3795011a8fae37480af5952e7bfface6f4a UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Updated] (FLINK-35835) SnapshotFileMergingCompatibilityITCase.testSwitchFromDisablingToEnablingFileMerging failed on AZP

2024-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-35835: --- Labels: pull-request-available (was: ) > SnapshotFileMergingCompatibilityITCase.testSwitchF

[PR] [FLINK-35835][test] Make file-merging test tolerate the scenario that source does not produce any record [flink]

2024-07-23 Thread via GitHub
Zakelly opened a new pull request, #25116: URL: https://github.com/apache/flink/pull/25116 ## What is the purpose of the change Like the title said, make file-merging test tolerate the scenario that source does not produce any record. ## Brief change log - Change the ve

[jira] [Updated] (FLINK-35874) Check pureBinlogPhaseTables set before call getBinlogPosition method in BinlogSplitReader

2024-07-23 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-35874: --- Affects Version/s: cdc-3.1.1 > Check pureBinlogPhaseTables set before call getBinlogPosition method

[jira] [Resolved] (FLINK-35874) Check pureBinlogPhaseTables set before call getBinlogPosition method in BinlogSplitReader

2024-07-23 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-35874. Resolution: Implemented Implemented via master(3.2-SNAPSHOT): ea71b2302ddc5f9b7be65843dbf3f5bed4ca

Re: [PR] [hotfix][docs] Fix MongoDB Driver version specification [flink-cdc]

2024-07-23 Thread via GitHub
Jiabao-Sun merged PR #3493: URL: https://github.com/apache/flink-cdc/pull/3493 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.

Re: [PR] [hotfix][docs] Fix MongoDB Driver version specification [flink-cdc]

2024-07-23 Thread via GitHub
Jiabao-Sun merged PR #3492: URL: https://github.com/apache/flink-cdc/pull/3492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.

[jira] [Updated] (FLINK-35874) Check pureBinlogPhaseTables set before call getBinlogPosition method in BinlogSplitReader

2024-07-23 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-35874: --- Fix Version/s: cdc-3.2.0 > Check pureBinlogPhaseTables set before call getBinlogPosition method in

Re: [PR] [FLINK-35874][cdc-connector][mysql] Check pureBinlogPhaseTables set before call getBinlogPosition method [flink-cdc]

2024-07-23 Thread via GitHub
leonardBang merged PR #3488: URL: https://github.com/apache/flink-cdc/pull/3488 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [FLINK-35857] fix redeploy failed deployment without latest checkpoint [flink-kubernetes-operator]

2024-07-23 Thread via GitHub
chenyuzhi459 commented on code in PR #855: URL: https://github.com/apache/flink-kubernetes-operator/pull/855#discussion_r1689050095 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/reconciler/deployment/AbstractJobReconciler.java: ## @@ -289,6 +28

Re: [PR] [FLINK-33874][runtime] Support resource request wait mechanism at DefaultDeclarativeSlotPool side for Default Scheduler [flink]

2024-07-23 Thread via GitHub
1996fanrui commented on code in PR #25113: URL: https://github.com/apache/flink/pull/25113#discussion_r1689012554 ## flink-runtime/src/main/java/org/apache/flink/runtime/jobmaster/slotpool/AbstractSlotPoolServiceFactory.java: ## @@ -34,14 +36,18 @@ public abstract class Abstract

[jira] [Commented] (FLINK-35885) proctime aggregate window triggered by watermark

2024-07-23 Thread Yun Tang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868232#comment-17868232 ] Yun Tang commented on FLINK-35885: -- I think this problem might be because user misuse e

Re: [PR] [FLINK-35822] Introduce DESCRIBE FUNCTION [flink]

2024-07-23 Thread via GitHub
flinkbot commented on PR #25115: URL: https://github.com/apache/flink/pull/25115#issuecomment-2246777977 ## CI report: * 70f38a3197427c5fec17f7c7a3578ed1291c97f8 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-23 Thread via GitHub
anupamaggarwal commented on PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#issuecomment-2246776962 Thanks for contributing this super useful feature @mateczagany 👍 !. I am still new to the operator codebase so pls feel free to ignore if something is not relev

[jira] [Updated] (FLINK-35822) FLIP-465: Introduce DESCRIBE FUNCTION

2024-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-35822: --- Labels: pull-request-available (was: ) > FLIP-465: Introduce DESCRIBE FUNCTION > --

[PR] [FLINK-35822] Introduce DESCRIBE FUNCTION [flink]

2024-07-23 Thread via GitHub
nateab opened a new pull request, #25115: URL: https://github.com/apache/flink/pull/25115 ## What is the purpose of the change This PR introduces `DESCRIBE FUNCTION` syntax, allowing users to describe the metadata of a given function. The design for this is covered under FLIP

Re: [PR] [hotfix][docs] Fix MongoDB Driver version specification [flink-cdc]

2024-07-23 Thread via GitHub
yuxiqian commented on code in PR #3493: URL: https://github.com/apache/flink-cdc/pull/3493#discussion_r1689044606 ## docs/content.zh/docs/connectors/flink-sources/overview.md: ## @@ -37,7 +37,7 @@ You can also read [tutorials]({{< ref "docs/connectors/flink-sources/tutorials/b

[jira] [Updated] (FLINK-35884) Pipeline MySQL connector supports setting chunk column key

2024-07-23 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-35884: --- Fix Version/s: cdc-3.2.0 > Pipeline MySQL connector supports setting chunk column key >

Re: [PR] [FLINK-35888][cdc-connector][paimon] Add e2e test for PaimonDataSink. [flink-cdc]

2024-07-23 Thread via GitHub
yuxiqian commented on code in PR #3491: URL: https://github.com/apache/flink-cdc/pull/3491#discussion_r1689033306 ## flink-cdc-e2e-tests/flink-cdc-pipeline-e2e-tests/pom.xml: ## @@ -113,6 +134,152 @@ limitations under the License. ${testcontainers.version}

[jira] [Updated] (FLINK-35885) proctime aggregate window triggered by watermark

2024-07-23 Thread Baozhu Zhao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baozhu Zhao updated FLINK-35885: Description: We have discovered an unexpected case where abnormal data with a count of 0 occurs w

[jira] [Updated] (FLINK-35885) proctime aggregate window triggered by watermark

2024-07-23 Thread Baozhu Zhao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baozhu Zhao updated FLINK-35885: Description: We have discovered an unexpected case where abnormal data with a count of 0 occurs w

[jira] [Commented] (FLINK-35835) SnapshotFileMergingCompatibilityITCase.testSwitchFromDisablingToEnablingFileMerging failed on AZP

2024-07-23 Thread Zakelly Lan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868229#comment-17868229 ] Zakelly Lan commented on FLINK-35835: - I was able to reproduce this when I add {{Thr

Re: [PR] [hotfix][docs] Fix MongoDB Driver version specification [flink-cdc]

2024-07-23 Thread via GitHub
Jiabao-Sun commented on code in PR #3492: URL: https://github.com/apache/flink-cdc/pull/3492#discussion_r1689036246 ## docs/content.zh/docs/connectors/flink-sources/overview.md: ## @@ -37,7 +37,7 @@ You can also read [tutorials]({{< ref "docs/connectors/flink-sources/tutorials/

Re: [PR] [hotfix][docs] Fix MongoDB Driver version specification [flink-cdc]

2024-07-23 Thread via GitHub
Jiabao-Sun commented on code in PR #3493: URL: https://github.com/apache/flink-cdc/pull/3493#discussion_r1689035628 ## docs/content.zh/docs/connectors/flink-sources/overview.md: ## @@ -37,7 +37,7 @@ You can also read [tutorials]({{< ref "docs/connectors/flink-sources/tutorials/

[jira] [Updated] (FLINK-35885) proctime aggregate window triggered by watermark

2024-07-23 Thread Baozhu Zhao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baozhu Zhao updated FLINK-35885: Description: We have discovered an unexpected case where abnormal data with a count of 0 occurs w

[jira] [Commented] (FLINK-35835) SnapshotFileMergingCompatibilityITCase.testSwitchFromDisablingToEnablingFileMerging failed on AZP

2024-07-23 Thread Zakelly Lan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868227#comment-17868227 ] Zakelly Lan commented on FLINK-35835: - [~fanrui] Thanks for the reminder! I'll take

[jira] [Updated] (FLINK-35876) mysql cdc may change from snapshot mode to binlog mode failed when cpu load of mysql server is 100%

2024-07-23 Thread Youle (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Youle updated FLINK-35876: -- Description: I use mysql cdc to read data from mysql to write to doris, mysql cdc  may change from sanpshot m

[jira] [Assigned] (FLINK-35884) Pipeline MySQL connector supports setting chunk column key

2024-07-23 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-35884: -- Assignee: JunboWang > Pipeline MySQL connector supports setting chunk column key > --

[jira] [Assigned] (FLINK-35888) Add e2e test for paimon DataSink

2024-07-23 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-35888: -- Assignee: LvYanquan > Add e2e test for paimon DataSink > > >

[jira] [Updated] (FLINK-35885) proctime aggregate window triggered by watermark

2024-07-23 Thread Baozhu Zhao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baozhu Zhao updated FLINK-35885: Summary: proctime aggregate window triggered by watermark (was: SlicingWindowOperator should not

Re: [PR] [FLINK-35888][cdc-connector][paimon] Add e2e test for PaimonDataSink. [flink-cdc]

2024-07-23 Thread via GitHub
lvyanquan commented on PR #3491: URL: https://github.com/apache/flink-cdc/pull/3491#issuecomment-2246724208 @yuxiqian Can you help to review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (FLINK-35876) mysql cdc may change from snapshot mode to binlog mode failed when cpu load of mysql server is 100%

2024-07-23 Thread Paul Lin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868224#comment-17868224 ] Paul Lin commented on FLINK-35876: -- +1. I've experienced a similar problem with CDC 3.0

[PR] [docs] Fix MongoDB Driver version specification [flink-cdc]

2024-07-23 Thread via GitHub
yuxiqian opened a new pull request, #3493: URL: https://github.com/apache/flink-cdc/pull/3493 This corrects MongoDB Driver version specification in docs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] [docs] Fix MongoDB Driver version specification [flink-cdc]

2024-07-23 Thread via GitHub
yuxiqian opened a new pull request, #3492: URL: https://github.com/apache/flink-cdc/pull/3492 This corrects MongoDB Driver version specification in docs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [FLINK-35868][cdc-connector][mongodb] Bump dependency version to support MongoDB 7.0 [flink-cdc]

2024-07-23 Thread via GitHub
Jiabao-Sun commented on PR #3489: URL: https://github.com/apache/flink-cdc/pull/3489#issuecomment-2246709941 Thanks @yuxiqian for the investigation. How about bump the driver version to 4.11? -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] [FLINK-33874][runtime] Support resource request wait mechanism at DefaultDeclarativeSlotPool side for Default Scheduler [flink]

2024-07-23 Thread via GitHub
1996fanrui commented on PR #25113: URL: https://github.com/apache/flink/pull/25113#issuecomment-2246699764 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[jira] [Commented] (FLINK-35835) SnapshotFileMergingCompatibilityITCase.testSwitchFromDisablingToEnablingFileMerging failed on AZP

2024-07-23 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868218#comment-17868218 ] Rui Fan commented on FLINK-35835: - Hi [~zakelly] , this CI failure happens on master bra

Re: [PR] [FLINK-35272][cdc][runtime] Pipeline Transform job supports omitting / renaming calculation column [flink-cdc]

2024-07-23 Thread via GitHub
yuxiqian commented on code in PR #3285: URL: https://github.com/apache/flink-cdc/pull/3285#discussion_r1688271544 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/operators/transform/TransformFilterProcessor.java: ## Review Comment: To make newly added E2e te

[jira] [Assigned] (FLINK-33977) Adaptive scheduler may not minimize the number of TMs during downscaling

2024-07-23 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Fan reassigned FLINK-33977: --- Assignee: RocMarshal > Adaptive scheduler may not minimize the number of TMs during downscaling > -

Re: [PR] [FLINK-35242] Supports per-SE type configuration & "lenient" evolution behavior [flink-cdc]

2024-07-23 Thread via GitHub
yuxiqian commented on code in PR #3339: URL: https://github.com/apache/flink-cdc/pull/3339#discussion_r1688992338 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/operators/schema/metrics/SchemaOperatorMetrics.java: ## @@ -0,0 +1,87 @@ +/* + * Licensed to the Apac

Re: [PR] [FLINK-35242] Supports per-SE type configuration & "lenient" evolution behavior [flink-cdc]

2024-07-23 Thread via GitHub
yuxiqian commented on code in PR #3339: URL: https://github.com/apache/flink-cdc/pull/3339#discussion_r1688991979 ## flink-cdc-connect/flink-cdc-pipeline-connectors/flink-cdc-pipeline-connector-doris/src/main/java/org/apache/flink/cdc/connectors/doris/sink/DorisDataSink.java: ##

Re: [PR] [FLINK-35242] Supports per-SE type configuration & "lenient" evolution behavior [flink-cdc]

2024-07-23 Thread via GitHub
yuxiqian commented on code in PR #3339: URL: https://github.com/apache/flink-cdc/pull/3339#discussion_r1688991587 ## flink-cdc-common/src/main/java/org/apache/flink/cdc/common/event/SchemaChangeEventTypeFamily.java: ## @@ -0,0 +1,77 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] [FLINK-35242] Supports per-SE type configuration & "lenient" evolution behavior [flink-cdc]

2024-07-23 Thread via GitHub
yuxiqian commented on code in PR #3339: URL: https://github.com/apache/flink-cdc/pull/3339#discussion_r1688989620 ## flink-cdc-common/src/main/java/org/apache/flink/cdc/common/event/SchemaChangeEventType.java: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] [FLINK-35242] Supports per-SE type configuration & "lenient" evolution behavior [flink-cdc]

2024-07-23 Thread via GitHub
yuxiqian commented on code in PR #3339: URL: https://github.com/apache/flink-cdc/pull/3339#discussion_r1688986645 ## flink-cdc-common/src/main/java/org/apache/flink/cdc/common/sink/MetadataApplier.java: ## @@ -19,13 +19,22 @@ import org.apache.flink.cdc.common.annotation.Publ

Re: [PR] [FLINK-35242] Supports per-SE type configuration & "lenient" evolution behavior [flink-cdc]

2024-07-23 Thread via GitHub
yuxiqian commented on code in PR #3339: URL: https://github.com/apache/flink-cdc/pull/3339#discussion_r1688985573 ## flink-cdc-common/src/main/java/org/apache/flink/cdc/common/sink/DataSink.java: ## @@ -30,5 +33,5 @@ public interface DataSink { EventSinkProvider getEventSin

Re: [PR] [enhancement] add param which contain column comments of source table field [flink-cdc]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #3039: URL: https://github.com/apache/flink-cdc/pull/3039#issuecomment-2246585165 This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs. --

Re: [PR] [FLINK-35421]fixed schema operator blocking when restart [flink-cdc]

2024-07-23 Thread via GitHub
github-actions[bot] commented on PR #3350: URL: https://github.com/apache/flink-cdc/pull/3350#issuecomment-2246584970 This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs. --

[jira] [Updated] (FLINK-35888) Add e2e test for paimon DataSink

2024-07-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-35888: --- Labels: pull-request-available (was: ) > Add e2e test for paimon DataSink > ---

[PR] [FLINK-35888][cdc-connector][paimon] Add e2e test for PaimonDataSink. [flink-cdc]

2024-07-23 Thread via GitHub
lvyanquan opened a new pull request, #3491: URL: https://github.com/apache/flink-cdc/pull/3491 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[jira] [Created] (FLINK-35888) Add e2e test for paimon DataSink

2024-07-23 Thread LvYanquan (Jira)
LvYanquan created FLINK-35888: - Summary: Add e2e test for paimon DataSink Key: FLINK-35888 URL: https://issues.apache.org/jira/browse/FLINK-35888 Project: Flink Issue Type: Technical Debt

Re: [PR] [FLINK-35305]Amazon SQS Sink Connector [flink-connector-aws]

2024-07-23 Thread via GitHub
19priyadhingra commented on code in PR #141: URL: https://github.com/apache/flink-connector-aws/pull/141#discussion_r1688281873 ## flink-connector-aws/flink-connector-sqs/src/test/java/org/apache/flink/connector/sqs/sink/SqsSinkBuilderTest.java: ## @@ -25,14 +25,6 @@ /** Cover

[jira] [Comment Edited] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-23 Thread Trystan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868193#comment-17868193 ] Trystan edited comment on FLINK-35285 at 7/23/24 9:39 PM: -- Grea

[jira] [Comment Edited] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-23 Thread Trystan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868193#comment-17868193 ] Trystan edited comment on FLINK-35285 at 7/23/24 9:31 PM: -- Grea

[jira] [Commented] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-23 Thread Trystan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868193#comment-17868193 ] Trystan commented on FLINK-35285: - Great, we'll experiment with setting the max parallel

Re: [PR] [FLINK-22748] Allow dynamic target topic selection in SQL Kafka sinks [flink-connector-kafka]

2024-07-23 Thread via GitHub
dmariassy commented on code in PR #109: URL: https://github.com/apache/flink-connector-kafka/pull/109#discussion_r1688736888 ## flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/table/DynamicKafkaRecordSerializationSchema.java: ## @@ -144,14 +147,3

Re: [PR] [DRAFT][FLINK-34440][formats][protobuf-confluent] Protobuf confluent dynamic format [flink]

2024-07-23 Thread via GitHub
flinkbot commented on PR #25114: URL: https://github.com/apache/flink/pull/25114#issuecomment-2246160358 ## CI report: * de53e4219ddb42d08a578989675ebec012dd0b35 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [DRAFT][FLINK-34440][formats][protobuf-confluent] add support for protobuf-confluent [flink]

2024-07-23 Thread via GitHub
dmariassy commented on PR #24482: URL: https://github.com/apache/flink/pull/24482#issuecomment-2246151636 Draft PR for the alternative implementation: https://github.com/apache/flink/pull/25114 -- This is an automated message from the Apache Git Service. To respond to the message, please

[PR] [DRAFT][FLINK-34440][formats][protobuf-confluent] Protobuf confluent dynamic format [flink]

2024-07-23 Thread via GitHub
dmariassy opened a new pull request, #25114: URL: https://github.com/apache/flink/pull/25114 **This is a DRAFT PR**. ## TODO - Add missing boilerplate (e.g. config) - Add debezium support - Test the format in Shopify Flink jobs - Docs (might be a separate PR) ##

[jira] [Commented] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-23 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868169#comment-17868169 ] Gyula Fora commented on FLINK-35285: It's much better to not scale down than to unde

[jira] [Commented] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-23 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868168#comment-17868168 ] Gyula Fora commented on FLINK-35285: {noformat} I would argue that a current paralle

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-23 Thread via GitHub
mateczagany commented on code in PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#discussion_r1688472333 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/service/AbstractFlinkService.java: ## @@ -716,6 +695,48 @@ public C

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-23 Thread via GitHub
mateczagany commented on code in PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#discussion_r1688463075 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/listener/AuditUtils.java: ## @@ -53,6 +62,16 @@ private static Stri

[jira] [Updated] (FLINK-35887) Null Pointer Exception in TypeExtractor.isRecord when trying to provide type info for interface

2024-07-23 Thread Jacob Jona Fahlenkamp (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacob Jona Fahlenkamp updated FLINK-35887: -- Priority: Major (was: Minor) > Null Pointer Exception in TypeExtractor.isReco

[jira] [Created] (FLINK-35887) Null Pointer Exception in TypeExtractor.isRecord when trying to provide type info for interface

2024-07-23 Thread Jacob Jona Fahlenkamp (Jira)
Jacob Jona Fahlenkamp created FLINK-35887: - Summary: Null Pointer Exception in TypeExtractor.isRecord when trying to provide type info for interface Key: FLINK-35887 URL: https://issues.apache.org/jira/bro

[jira] [Comment Edited] (FLINK-35886) Incorrect watermark idleness timeout accounting when subtask is backpressured/blocked

2024-07-23 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868098#comment-17868098 ] Piotr Nowojski edited comment on FLINK-35886 at 7/23/24 3:45 PM: -

Re: [PR] [FLINK-35305]Amazon SQS Sink Connector [flink-connector-aws]

2024-07-23 Thread via GitHub
19priyadhingra commented on code in PR #141: URL: https://github.com/apache/flink-connector-aws/pull/141#discussion_r1688281873 ## flink-connector-aws/flink-connector-sqs/src/test/java/org/apache/flink/connector/sqs/sink/SqsSinkBuilderTest.java: ## @@ -25,14 +25,6 @@ /** Cover

Re: [PR] [FLINK-35272][cdc][runtime] Pipeline Transform job supports omitting / renaming calculation column [flink-cdc]

2024-07-23 Thread via GitHub
yuxiqian commented on code in PR #3285: URL: https://github.com/apache/flink-cdc/pull/3285#discussion_r1688271544 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/operators/transform/TransformFilterProcessor.java: ## Review Comment: I modified expression eval

Re: [PR] [FLINK-35272][cdc][runtime] Pipeline Transform job supports omitting / renaming calculation column [flink-cdc]

2024-07-23 Thread via GitHub
yuxiqian commented on code in PR #3285: URL: https://github.com/apache/flink-cdc/pull/3285#discussion_r1688271544 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/operators/transform/TransformFilterProcessor.java: ## Review Comment: I modified expression eval

[jira] [Commented] (FLINK-35886) Incorrect watermark idleness timeout accounting when subtask is backpressured/blocked

2024-07-23 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868098#comment-17868098 ] Piotr Nowojski commented on FLINK-35886: I think this problem requires a public

[jira] [Assigned] (FLINK-35886) Incorrect watermark idleness timeout accounting when subtask is backpressured/blocked

2024-07-23 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Nowojski reassigned FLINK-35886: -- Assignee: Piotr Nowojski > Incorrect watermark idleness timeout accounting when subta

[jira] [Created] (FLINK-35886) Incorrect watermark idleness timeout accounting when subtask is backpressured/blocked

2024-07-23 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-35886: -- Summary: Incorrect watermark idleness timeout accounting when subtask is backpressured/blocked Key: FLINK-35886 URL: https://issues.apache.org/jira/browse/FLINK-35886

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-23 Thread via GitHub
mateczagany commented on code in PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#discussion_r1688262282 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/controller/FlinkStateSnapshotContext.java: ## @@ -0,0 +1,112 @@ +/*

Re: [PR] [FLINK-22748][connector-kafka] Allow dynamic target topic selection in SQL Kafka sinks [flink-connector-kafka]

2024-07-23 Thread via GitHub
klam-shop commented on code in PR #109: URL: https://github.com/apache/flink-connector-kafka/pull/109#discussion_r1688236979 ## flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/table/DynamicKafkaRecordSerializationSchema.java: ## @@ -144,14 +147,3

[jira] [Comment Edited] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-23 Thread Trystan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868093#comment-17868093 ] Trystan edited comment on FLINK-35285 at 7/23/24 3:03 PM: -- {nof

[jira] [Commented] (FLINK-22748) Allow dynamic target topic selection in SQL Kafka sinks

2024-07-23 Thread Kevin Lam (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868094#comment-17868094 ] Kevin Lam commented on FLINK-22748: --- Hi I've taken some time to build upon [~nicholasj

[jira] [Comment Edited] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-23 Thread Trystan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868093#comment-17868093 ] Trystan edited comment on FLINK-35285 at 7/23/24 3:03 PM: -- {nof

[jira] [Comment Edited] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-23 Thread Trystan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868093#comment-17868093 ] Trystan edited comment on FLINK-35285 at 7/23/24 3:02 PM: -- {nof

Re: [PR] [FLINK-22748][connector-kafka] Allow dynamic target topic selection in SQL Kafka sinks [flink]

2024-07-23 Thread via GitHub
klam-shop commented on PR #16142: URL: https://github.com/apache/flink/pull/16142#issuecomment-2245500551 Hi I've taken some time to build upon [Nicholas Jiang](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=nicholasjiang) 's PR and port it to the new kafka connector repo:

[jira] [Comment Edited] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-23 Thread Trystan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868093#comment-17868093 ] Trystan edited comment on FLINK-35285 at 7/23/24 3:00 PM: -- {nof

[PR] [FLINK-22748][connector-kafka] Allow dynamic target topic selection in SQL Kafka sinks [flink-connector-kafka]

2024-07-23 Thread via GitHub
klam-shop opened a new pull request, #109: URL: https://github.com/apache/flink-connector-kafka/pull/109 ## What is the purpose of the change Allows writing to different Kafka topics based on the `topic` metadata column value in SQL, and updates the Table API's `KafkaDynamicSink` to accep

[jira] [Commented] (FLINK-35285) Autoscaler key group optimization can interfere with scale-down.max-factor

2024-07-23 Thread Trystan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868093#comment-17868093 ] Trystan commented on FLINK-35285: -   {noformat} As long as your job parallelism is very

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-23 Thread via GitHub
mateczagany commented on code in PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#discussion_r1688221106 ## flink-kubernetes-operator-api/src/main/java/org/apache/flink/kubernetes/operator/api/status/FlinkStateSnapshotState.java: ## @@ -0,0 +1,39 @@ +/

Re: [PR] [FLINK-35272][cdc][runtime] Pipeline Transform job supports omitting / renaming calculation column [flink-cdc]

2024-07-23 Thread via GitHub
yuxiqian commented on PR #3285: URL: https://github.com/apache/flink-cdc/pull/3285#issuecomment-2245470485 Done, rebased due to some conflicts with https://github.com/apache/flink-cdc/commit/26ff6d2a081181f3df7aa49d65d804c57c634122. Will add `CAST ... AS` tests after #3357 got merged. --

Re: [PR] [FLINK-35857] fix redeploy failed deployment without latest checkpoint [flink-kubernetes-operator]

2024-07-23 Thread via GitHub
gyfora commented on code in PR #855: URL: https://github.com/apache/flink-kubernetes-operator/pull/855#discussion_r1688196270 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/reconciler/deployment/AbstractJobReconciler.java: ## @@ -289,6 +289,8 @@

Re: [PR] [FLINK-35857] fix redeploy failed deployment without latest checkpoint [flink-kubernetes-operator]

2024-07-23 Thread via GitHub
chenyuzhi459 commented on code in PR #855: URL: https://github.com/apache/flink-kubernetes-operator/pull/855#discussion_r1688138227 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/reconciler/deployment/AbstractJobReconciler.java: ## @@ -289,6 +28

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-23 Thread via GitHub
mateczagany commented on code in PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#discussion_r1688136504 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/observer/SnapshotObserver.java: ## @@ -273,15 +280,19 @@ private vo

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-23 Thread via GitHub
mateczagany commented on code in PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#discussion_r1688126716 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/controller/FlinkStateSnapshotController.java: ## @@ -0,0 +1,216 @@

[jira] [Closed] (FLINK-35292) Set dummy savepoint path during last-state upgrade

2024-07-23 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gyula Fora closed FLINK-35292. -- Resolution: Fixed merged to main 207b149f9b556d68c4ab98d16cdde2f7820659c0 > Set dummy savepoint path

Re: [PR] [FLINK-35292] Set dummy savepoint path during last-state upgrade [flink-kubernetes-operator]

2024-07-23 Thread via GitHub
gyfora commented on code in PR #849: URL: https://github.com/apache/flink-kubernetes-operator/pull/849#discussion_r1688103240 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/reconciler/deployment/AbstractJobReconciler.java: ## @@ -265,7 +274,7 @@

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-23 Thread via GitHub
mateczagany commented on code in PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#discussion_r1688104509 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/controller/FlinkStateSnapshotContext.java: ## @@ -0,0 +1,112 @@ +/*

Re: [PR] [FLINK-35265] Implement FlinkStateSnapshot custom resource [flink-kubernetes-operator]

2024-07-23 Thread via GitHub
mateczagany commented on code in PR #821: URL: https://github.com/apache/flink-kubernetes-operator/pull/821#discussion_r1688100790 ## flink-kubernetes-operator/src/main/java/org/apache/flink/kubernetes/operator/config/KubernetesOperatorConfigOptions.java: ## @@ -426,6 +438,14 @

[jira] [Updated] (FLINK-35885) SlicingWindowOperator should not process watermark with proctime

2024-07-23 Thread Baozhu Zhao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baozhu Zhao updated FLINK-35885: Description: We have discovered an unexpected case where abnormal data with a count of 0 occurs w

Re: [PR] [FLINK-32682] Make it possible to use query time based time functions in streaming mode [flink]

2024-07-23 Thread via GitHub
dawidwys closed pull request #23083: [FLINK-32682] Make it possible to use query time based time functions in streaming mode URL: https://github.com/apache/flink/pull/23083 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

  1   2   >