[jira] [Commented] (FLINK-33555) LocalRecoveryITCase.testRecoverLocallyFromProcessCrashWithWorkingDirectory:

2024-09-22 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883672#comment-17883672 ] Weijie Guo commented on FLINK-33555: https://dev.azure.com/apache-flink/apache-flink

Re: [PR] [cdc-connector][cdc-base] Add SNAPSHOT mode [flink-cdc]

2024-09-22 Thread via GitHub
loserwang1024 commented on PR #2901: URL: https://github.com/apache/flink-cdc/pull/2901#issuecomment-2367145529 > Not supporting BATCH execution mode hi, @LinMingQiang, still not support execution.runtime-mode=BATCH now, though snapshot mode will read in batch. Because we still don't su

Re: [PR] [FLINK-36338] Properly handle KeyContext when using AsyncKeyedStateBackendAdaptor [flink]

2024-09-22 Thread via GitHub
fredia commented on code in PR #25367: URL: https://github.com/apache/flink/pull/25367#discussion_r1770726474 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/AsyncExecutionController.java: ## @@ -189,6 +194,9 @@ public RecordContext buildContext(Object re

[jira] [Commented] (FLINK-36166) testJoinDisorderChangeLog failed on AZP

2024-09-22 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883670#comment-17883670 ] Weijie Guo commented on FLINK-36166: https://dev.azure.com/apache-flink/apache-flink

[jira] [Commented] (FLINK-36290) OutOfMemoryError in connect test run

2024-09-22 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883671#comment-17883671 ] Weijie Guo commented on FLINK-36290: https://dev.azure.com/apache-flink/apache-flink

[jira] (FLINK-36166) testJoinDisorderChangeLog failed on AZP

2024-09-22 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36166 ] Weijie Guo deleted comment on FLINK-36166: was (Author: weijie guo): https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=62326&view=logs&j=1c002d28-a73d-5309-26ee-10036d847

[jira] [Updated] (FLINK-36348) Netty shuffle direct memory consumption end-to-end test failed due to direct memory OOM

2024-09-22 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-36348: --- Description: Found the root cause from downloaded artifacts. {code:java} org.apache.flink.runtime.io

[jira] [Updated] (FLINK-36348) Netty shuffle direct memory consumption end-to-end test failed due to direct memory OOM

2024-09-22 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-36348: --- Description: {code:java} org.apache.flink.runtime.io.network.netty.exception.LocalTransportExceptio

[jira] [Created] (FLINK-36348) Netty shuffle direct memory consumption end-to-end test failed due to direct memory OOM

2024-09-22 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-36348: -- Summary: Netty shuffle direct memory consumption end-to-end test failed due to direct memory OOM Key: FLINK-36348 URL: https://issues.apache.org/jira/browse/FLINK-36348 P

Re: [PR] [FLINK-36338] Properly handle KeyContext when using AsyncKeyedStateBackendAdaptor [flink]

2024-09-22 Thread via GitHub
Zakelly commented on code in PR #25367: URL: https://github.com/apache/flink/pull/25367#discussion_r1770742198 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/AsyncExecutionController.java: ## @@ -189,6 +194,9 @@ public RecordContext buildContext(Object r

Re: [PR] [FLINK-36338] Properly handle KeyContext when using AsyncKeyedStateBackendAdaptor [flink]

2024-09-22 Thread via GitHub
Zakelly commented on code in PR #25367: URL: https://github.com/apache/flink/pull/25367#discussion_r1770746753 ## flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/AsyncExecutionController.java: ## @@ -189,6 +194,9 @@ public RecordContext buildContext(Object r

Re: [PR] [FLINK-36321][state/forst] Execute read/write state request in different executors [flink]

2024-09-22 Thread via GitHub
fredia merged PR #25360: URL: https://github.com/apache/flink/pull/25360 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

Re: [PR] [Flink 36245] Remove legacy SourceFunction / SinkFunction / Sink V1 API in 2.0 [flink]

2024-09-22 Thread via GitHub
lvyanquan commented on PR #25331: URL: https://github.com/apache/flink/pull/25331#issuecomment-2367194793 Hi @PatrickRen, could you please help to review this as I'm not really familiar with the API norms. -- This is an automated message from the Apache Git Service. To respond to the mess

Re: [PR] [FLINK-35165][runtime/coordination] AdaptiveBatch Scheduler should not restrict the default source parall… [flink]

2024-09-22 Thread via GitHub
venkata91 commented on PR #24736: URL: https://github.com/apache/flink/pull/24736#issuecomment-2367194478 @JunRuiLee First of all, apologies for very late reply! I think the issue is, if source parallelism is not explicitly set and if `execution.batch.adaptive.auto-parallelism.max-par

[jira] [Resolved] (FLINK-36321) Execute read/write state request in different executor

2024-09-22 Thread Yanfei Lei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanfei Lei resolved FLINK-36321. Assignee: Yanfei Lei Resolution: Fixed Merged into master via 1d1bf5c8f > Execute read/writ

Re: [PR] [FLINK-36248][table] Introduce new Join Operator with Async State API [flink]

2024-09-22 Thread via GitHub
xuyangzhong commented on code in PR #25320: URL: https://github.com/apache/flink/pull/25320#discussion_r1770706342 ## flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/runtime/stream/sql/JoinITCase.scala: ## @@ -99,6 +106,33 @@ class JoinITCase(miniBa

Re: [PR] [FLINK-33977][runtime] Adaptive scheduler may not minimize the number of TMs during downscaling [flink]

2024-09-22 Thread via GitHub
1996fanrui commented on code in PR #25218: URL: https://github.com/apache/flink/pull/25218#discussion_r1770708552 ## flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/adaptive/allocator/SlotAssignerTest.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Softw

Re: [PR] [FLINK-33977][runtime] Adaptive scheduler may not minimize the number of TMs during downscaling [flink]

2024-09-22 Thread via GitHub
1996fanrui commented on code in PR #25218: URL: https://github.com/apache/flink/pull/25218#discussion_r1770708552 ## flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/adaptive/allocator/SlotAssignerTest.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Softw

Re: [PR] [FLINK-36335][runtime] Improving Method Reusability in StreamGraphGenerator with JobVertexBuildContext [flink]

2024-09-22 Thread via GitHub
JunRuiLee commented on code in PR #25366: URL: https://github.com/apache/flink/pull/25366#discussion_r1768349683 ## flink-runtime/src/main/java/org/apache/flink/streaming/api/graph/util/ChainedOperatorHashInfo.java: ## @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Fou

[jira] [Updated] (FLINK-36347) Using the offset obtained after a query transaction as a high watermark cannot ensure exactly-once semantics.

2024-09-22 Thread Zhongmin Qiao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongmin Qiao updated FLINK-36347: -- External issue URL: https://github.com/apache/flink-cdc/pull/3616 > Using the offset obtained

[jira] [Commented] (FLINK-33750) Remove deprecated config options.

2024-09-22 Thread Xuannan Su (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883652#comment-17883652 ] Xuannan Su commented on FLINK-33750: Most easy-to-remove ConfigOptions are removed

Re: [PR] [FLINK-36338] Properly handle KeyContext when using AsyncKeyedStateBackendAdaptor [flink]

2024-09-22 Thread via GitHub
Zakelly commented on code in PR #25367: URL: https://github.com/apache/flink/pull/25367#discussion_r1770772924 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/AbstractKeyedStateBackend.java: ## @@ -250,6 +250,13 @@ public void setCurrentKey(K newKey) {

Re: [PR] [FLINK-33977][runtime] Adaptive scheduler may not minimize the number of TMs during downscaling [flink]

2024-09-22 Thread via GitHub
RocMarshal commented on code in PR #25218: URL: https://github.com/apache/flink/pull/25218#discussion_r1770790841 ## flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/adaptive/allocator/SlotAssignerTest.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Softw

Re: [PR] [FLINK-36273][table] Remove deprecated Table/SQL configuration in 2.0 [flink]

2024-09-22 Thread via GitHub
xuyangzhong commented on PR #25334: URL: https://github.com/apache/flink/pull/25334#issuecomment-2367274893 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] [FLINK-33678][streaming-java] Remove CheckpointStorage related public API. [flink]

2024-09-22 Thread via GitHub
JunRuiLee closed pull request #25264: [FLINK-33678][streaming-java] Remove CheckpointStorage related public API. URL: https://github.com/apache/flink/pull/25264 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] [FLINK-33977][runtime] Adaptive scheduler may not minimize the number of TMs during downscaling [flink]

2024-09-22 Thread via GitHub
XComp commented on PR #25218: URL: https://github.com/apache/flink/pull/25218#issuecomment-2367315487 Sure, I can take a pass over it today. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] [FLINK-36175][python] Remove `register_table_source` and `register_table_sink` in python module [flink]

2024-09-22 Thread via GitHub
xuyangzhong commented on PR #25266: URL: https://github.com/apache/flink/pull/25266#issuecomment-2367368865 The failed test is traced by https://issues.apache.org/jira/browse/FLINK-36293 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] [FLINK-36257][API] Remove easy-to-drop deprecated API [flink]

2024-09-22 Thread via GitHub
reswqa commented on PR #25347: URL: https://github.com/apache/flink/pull/25347#issuecomment-2367285342 Thanks for the review, merging... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] [FLINK-36257][API] Remove easy-to-drop deprecated API [flink]

2024-09-22 Thread via GitHub
reswqa merged PR #25347: URL: https://github.com/apache/flink/pull/25347 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

Re: [PR] [FLINK-36251][streaming-java] Remove StateBackend-related configuration getters/setters that return/set complex Java objects. [flink]

2024-09-22 Thread via GitHub
zhuzhurk commented on code in PR #25352: URL: https://github.com/apache/flink/pull/25352#discussion_r1768319833 ## flink-state-backends/flink-statebackend-changelog/src/test/java/org/apache/flink/state/changelog/ChangelogStateBackendLoadingTest.java: ## @@ -219,29 +218,14 @@ pub

[jira] [Resolved] (FLINK-36257) Remove easy-to-drop deprecated APIs

2024-09-22 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo resolved FLINK-36257. Fix Version/s: 2.0-preview Resolution: Done master(2.0-preview) via 7989f485f109d213dfc5a0d

Re: [PR] [FLINK-36251][streaming-java] Remove StateBackend-related configuration getters/setters that return/set complex Java objects. [flink]

2024-09-22 Thread via GitHub
JunRuiLee commented on PR #25352: URL: https://github.com/apache/flink/pull/25352#issuecomment-2367290298 Thanks @zhuzhurk , I've updated this pr. PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [FLINK-35165][runtime/coordination] AdaptiveBatch Scheduler should not restrict the default source parall… [flink]

2024-09-22 Thread via GitHub
venkata91 commented on PR #24736: URL: https://github.com/apache/flink/pull/24736#issuecomment-2367214026 To illustrate with an example test case, try changing the `executeJob` and `createConfiguration` with below code and run `testScheduling`: ``` private void executeJob(Boolean u

Re: [PR] [FLINK-36188] Fix disable buffer flush lose efficacy [flink-connector-hbase]

2024-09-22 Thread via GitHub
MOBIN-F commented on PR #49: URL: https://github.com/apache/flink-connector-hbase/pull/49#issuecomment-2367334732 @MartijnVisser can you help to trigger a CI run, thanks~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[jira] [Commented] (FLINK-33555) LocalRecoveryITCase.testRecoverLocallyFromProcessCrashWithWorkingDirectory:

2024-09-22 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883710#comment-17883710 ] Matthias Pohl commented on FLINK-33555: --- https://dev.azure.com/apache-flink/apache

[jira] [Commented] (FLINK-34447) ActiveResourceManagerTest#testWorkerRegistrationTimeoutNotCountingAllocationTime still fails on slow machines

2024-09-22 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883711#comment-17883711 ] Matthias Pohl commented on FLINK-34447: --- https://dev.azure.com/apache-flink/apache

[PR] [FLINK-36346][API] Remove deprecated API in flink-streaming-java module [flink]

2024-09-22 Thread via GitHub
reswqa opened a new pull request, #25374: URL: https://github.com/apache/flink/pull/25374 ## What is the purpose of the change *Remove deprecated API in flink-streaming-java module* ## Brief change log - *Remove deprecated API in flink-streaming-java module*

[jira] [Updated] (FLINK-36346) Remove deprecated API in flink-streaming-java module

2024-09-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-36346: --- Labels: pull-request-available (was: ) > Remove deprecated API in flink-streaming-java modu

Re: [PR] [FLINK-36346][API] Remove deprecated API in flink-streaming-java module [flink]

2024-09-22 Thread via GitHub
flinkbot commented on PR #25374: URL: https://github.com/apache/flink/pull/25374#issuecomment-2366135001 ## CI report: * 530bf52976a08ad3b847e6c1edd2b0304e329090 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Created] (FLINK-36347) Using the offset obtained after a query transaction as a high watermark cannot ensure exactly-once semantics.

2024-09-22 Thread Zhongmin Qiao (Jira)
Zhongmin Qiao created FLINK-36347: - Summary: Using the offset obtained after a query transaction as a high watermark cannot ensure exactly-once semantics. Key: FLINK-36347 URL: https://issues.apache.org/jira/brows

[jira] [Updated] (FLINK-36347) Using the offset obtained after a query transaction as a high watermark cannot ensure exactly-once semantics.

2024-09-22 Thread Zhongmin Qiao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongmin Qiao updated FLINK-36347: -- Description: !image-2024-09-22-16-51-09-930.png|width=891,height=564! Using the offset obtain

[PR] [FLINK-35874][cdc-connector][mysql] Obtain the high watermark offset and table data in the same transaction during the Snapshot phase [flink-cdc]

2024-09-22 Thread via GitHub
qiaozongmi opened a new pull request, #3616: URL: https://github.com/apache/flink-cdc/pull/3616 Obtain the high watermark offset and table data in the same transaction during the Snapshot phase,This ensures that the data retrieved is consistent and avoids any potential data loss. -- This

Re: [PR] [FLINK-36292][Connectors/Common]: remove timeout to avoid timeout exception [flink]

2024-09-22 Thread via GitHub
showuon commented on PR #25371: URL: https://github.com/apache/flink/pull/25371#issuecomment-2365680553 @becketqin , please take a look. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] [FLINK-29741] Remove flink-scala and flink-streaming-scala dependency exclude table module [flink]

2024-09-22 Thread via GitHub
reswqa merged PR #25324: URL: https://github.com/apache/flink/pull/25324 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

Re: [PR] [FLINK-36310] Remove per-job and run-application [flink]

2024-09-22 Thread via GitHub
reswqa merged PR #25348: URL: https://github.com/apache/flink/pull/25348 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

[jira] [Created] (FLINK-36346) Remove deprecated API in flink-streaming-java module

2024-09-22 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-36346: -- Summary: Remove deprecated API in flink-streaming-java module Key: FLINK-36346 URL: https://issues.apache.org/jira/browse/FLINK-36346 Project: Flink Issue Type:

[jira] [Resolved] (FLINK-36347) Using the offset obtained after a query transaction as a high watermark cannot ensure exactly-once semantics.

2024-09-22 Thread Zhongmin Qiao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongmin Qiao resolved FLINK-36347. --- Resolution: Fixed > Using the offset obtained after a query transaction as a high watermark

[jira] [Commented] (FLINK-36347) Using the offset obtained after a query transaction as a high watermark cannot ensure exactly-once semantics.

2024-09-22 Thread Zhongmin Qiao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883579#comment-17883579 ] Zhongmin Qiao commented on FLINK-36347: --- Assign this pr to me please.  [ https://g

[jira] [Assigned] (FLINK-36295) AdaptiveSchedulerClusterITCase. testCheckpointStatsPersistedAcrossRescale failed with

2024-09-22 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl reassigned FLINK-36295: - Assignee: Zdenek Tison (was: Matthias Pohl) > AdaptiveSchedulerClusterITCase. testChec

Re: [PR] [FLINK-35780] Support state migration between disabling and enabling State TTL for RocksDBState [flink]

2024-09-22 Thread via GitHub
xiangyuf commented on PR #25035: URL: https://github.com/apache/flink/pull/25035#issuecomment-2366757734 @Zakelly Currently, the serializer's compatibility check assumes the new serializer and previous serializer should have the same datatype. This does not apply to compatibility check betw

Re: [PR] [FLINK-36248][table] Introduce new Join Operator with Async State API [flink]

2024-09-22 Thread via GitHub
xishuaidelin commented on code in PR #25320: URL: https://github.com/apache/flink/pull/25320#discussion_r1770576947 ## flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/runtime/stream/sql/JoinITCase.scala: ## @@ -99,6 +106,33 @@ class JoinITCase(miniB