Re: [PR] [FLINK-37634][pipeline-connector/mysql]When the table name of mysql s… [flink-cdc]

2025-04-21 Thread via GitHub
linjianchang commented on PR #3996: URL: https://github.com/apache/flink-cdc/pull/3996#issuecomment-2817800574 > As documented in the official MySQL 8.4 reference manual: https://dev.mysql.com/doc/refman/8.4/en/identifier-case-sensitivity.html The case sensitivity behavior of MySQL identifi

Re: [PR] [FLINK-35093][postgres] Add COMMITTED_OFFSET startup mode for Postgres CDC to resume from existing replication slot [flink-cdc]

2025-04-21 Thread via GitHub
loserwang1024 commented on code in PR #3950: URL: https://github.com/apache/flink-cdc/pull/3950#discussion_r2052058018 ## flink-cdc-connect/flink-cdc-source-connectors/flink-connector-postgres-cdc/src/test/java/org/apache/flink/cdc/connectors/postgres/table/PostgreSQLConnectorITCase.jav

[jira] [Updated] (FLINK-37285) Adapt DataSinkTranslator to support new Sink API of FLIP-372

2025-04-21 Thread Yanquan Lv (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanquan Lv updated FLINK-37285: --- Fix Version/s: (was: cdc-3.4.0) > Adapt DataSinkTranslator to support new Sink API of FLIP-372 >

Re: [PR] [BP-1.20][FLINK-37197][docs] Update MongoDB docs for 2.0.0 release [flink]

2025-04-21 Thread via GitHub
leonardBang merged PR #26480: URL: https://github.com/apache/flink/pull/26480 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.a

[jira] [Created] (FLINK-37707) Deprecate DeserializationSchema#deserialize(byte[])

2025-04-21 Thread Hongjia Liang (Jira)
Hongjia Liang created FLINK-37707: - Summary: Deprecate DeserializationSchema#deserialize(byte[]) Key: FLINK-37707 URL: https://issues.apache.org/jira/browse/FLINK-37707 Project: Flink Issue

[jira] [Updated] (FLINK-37707) Deprecate DeserializationSchema#deserialize(byte[])

2025-04-21 Thread Hongjia Liang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongjia Liang updated FLINK-37707: -- Component/s: API / Type Serialization System (was: Formats (JSON, Avro, Pa

[jira] [Updated] (FLINK-37707) Deprecate DeserializationSchema#deserialize(byte[])

2025-04-21 Thread Hongjia Liang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongjia Liang updated FLINK-37707: -- Component/s: Table SQL / API (was: API / Type Serialization System) > De

[jira] [Updated] (FLINK-37707) Deprecate DeserializationSchema#deserialize(byte[])

2025-04-21 Thread Hongjia Liang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongjia Liang updated FLINK-37707: -- Description: Deprecate method `DeserializationSchema#deserialize(byte[])` and use `Deserializ

Re: [PR] [FLINK-37705] JobPlanInfo properties spec adjustments [flink]

2025-04-21 Thread via GitHub
Efrat19 commented on PR #26488: URL: https://github.com/apache/flink/pull/26488#issuecomment-2817914260 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [FLINK-36793][cdc-source-connectors] Fix the problem with the block splitter logic of Oracle CDC incremental snapshot, causing the split slice to be too large [flink-cdc]

2025-04-21 Thread via GitHub
linjianchang commented on code in PR #3841: URL: https://github.com/apache/flink-cdc/pull/3841#discussion_r2052128909 ## flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base/src/main/java/org/apache/flink/cdc/connectors/base/source/assigner/splitter/JdbcSourceChunkSplitter.java:

[jira] [Assigned] (FLINK-37578) CDC Pipeline distributed schema evolution topology with bad internal state

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-37578: -- Assignee: yux > CDC Pipeline distributed schema evolution topology with bad internal state >

[jira] [Updated] (FLINK-37578) CDC Pipeline distributed schema evolution topology with bad internal state

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-37578: --- Fix Version/s: cdc-3.4.0 > CDC Pipeline distributed schema evolution topology with bad internal stat

[jira] [Resolved] (FLINK-37578) CDC Pipeline distributed schema evolution topology with bad internal state

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-37578. Resolution: Implemented Fixed via master: 47433993575d222808eb9cabe17b63c79270ae3d > CDC Pipeline

Re: [PR] [FLINK-37578] Fix distributed schema registry exposes bad internal state accidentally [flink-cdc]

2025-04-21 Thread via GitHub
leonardBang merged PR #3972: URL: https://github.com/apache/flink-cdc/pull/3972 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [hotfix] sqlserver limit statement support [flink-connector-jdbc]

2025-04-21 Thread via GitHub
eskabetxe commented on code in PR #160: URL: https://github.com/apache/flink-connector-jdbc/pull/160#discussion_r2052134266 ## flink-connector-jdbc-core/src/main/java/org/apache/flink/connector/jdbc/core/database/dialect/JdbcDialect.java: ## @@ -58,6 +58,16 @@ public interface

Re: [PR] [FLINK-36793][cdc-source-connectors] Fix the problem with the block splitter logic of Oracle CDC incremental snapshot, causing the split slice to be too large [flink-cdc]

2025-04-21 Thread via GitHub
leonardBang commented on PR #3841: URL: https://github.com/apache/flink-cdc/pull/3841#issuecomment-2817941698 @linjianchang Thanks for your update, could you rebase to latest master branch which has been migrated to junit5 for all tests? -- This is an automated message from the Apache Git

Re: [PR] [hotfix] [base] Fix the serialization error occurring with table names containing dot [flink-cdc]

2025-04-21 Thread via GitHub
leonardBang commented on code in PR #2705: URL: https://github.com/apache/flink-cdc/pull/2705#discussion_r2053300902 ## flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base/src/test/java/org/apache/flink/cdc/connectors/base/source/meta/split/SourceSplitSerializerTest.java: #

[jira] [Updated] (FLINK-37712) TableNotFound Exception when reading ParallelMetadataSource to Paimon Sink

2025-04-21 Thread Yanquan Lv (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanquan Lv updated FLINK-37712: --- Summary: TableNotFound Exception when reading ParallelMetadataSource to Paimon Sink (was: Table) >

[jira] [Updated] (FLINK-37712) Met TableNotFound exception when reading ParallelMetadataSource to Paimon Sink

2025-04-21 Thread Yanquan Lv (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanquan Lv updated FLINK-37712: --- Summary: Met TableNotFound exception when reading ParallelMetadataSource to Paimon Sink (was: Table

Re: [PR] [FLINK-37697][runtime] Fix application mode cluster shut down before … [flink]

2025-04-21 Thread via GitHub
JunRuiLee commented on code in PR #26489: URL: https://github.com/apache/flink/pull/26489#discussion_r2053295509 ## flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/Dispatcher.java: ## @@ -1256,7 +1268,8 @@ private void setClientHeartbeatTimeoutForInitializedJob()

[jira] [Updated] (FLINK-37712) Met TableNotFound exception when reading ParallelMetadataSource to Paimon Sink

2025-04-21 Thread Yanquan Lv (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanquan Lv updated FLINK-37712: --- Description: when using ParallelMetadataSource as Source, the Flink topology will be like: ---> 1.

[jira] [Updated] (FLINK-37712) Met TableNotFound exception when reading ParallelMetadataSource to Paimon Sink

2025-04-21 Thread Yanquan Lv (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanquan Lv updated FLINK-37712: --- Component/s: Flink CDC Affects Version/s: cdc-3.4.0 Description: when using Para

Re: [PR] [FLINK-35093][postgres] Add COMMITTED_OFFSET startup mode for Postgres CDC to resume from existing replication slot [flink-cdc]

2025-04-21 Thread via GitHub
loserwang1024 commented on code in PR #3950: URL: https://github.com/apache/flink-cdc/pull/3950#discussion_r2053292573 ## flink-cdc-connect/flink-cdc-source-connectors/flink-connector-postgres-cdc/src/test/java/org/apache/flink/cdc/connectors/postgres/table/PostgreSQLConnectorITCase.jav

[jira] [Created] (FLINK-37712) Table

2025-04-21 Thread Yanquan Lv (Jira)
Yanquan Lv created FLINK-37712: -- Summary: Table Key: FLINK-37712 URL: https://issues.apache.org/jira/browse/FLINK-37712 Project: Flink Issue Type: Bug Reporter: Yanquan Lv -- T

Re: [PR] [hotfix] Rename SessionDispatcherFactory to DefaultDispatcherFactory [flink]

2025-04-21 Thread via GitHub
beliefer commented on PR #26490: URL: https://github.com/apache/flink/pull/26490#issuecomment-2820015112 ping @tillrohrmann @zentol @kl0u @XComp @JunRuiLee cc @1996fanrui @davidradl @GOODBOY008 -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Resolved] (FLINK-37706) [Flink CDC] Oceanbase OceanBaseMetadataApplierTest failed due to Caused by: java.sql.SQLException: Server out of disk space

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-37706. Resolution: Fixed via master: 4cf54c61f9f9283ba1138ffc0179f53f6b67ce3f > [Flink CDC] Oceanbase Oc

Re: [PR] [FLINK-37706][source-connector/oceanbase][tests] Use lts docker image version instead of latest for test [flink-cdc]

2025-04-21 Thread via GitHub
leonardBang merged PR #3998: URL: https://github.com/apache/flink-cdc/pull/3998 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

[PR] [FLINK-37711] Remove unused getJobGraph from PipelineExecutorUtils [flink]

2025-04-21 Thread via GitHub
beliefer opened a new pull request, #26493: URL: https://github.com/apache/flink/pull/26493 ## What is the purpose of the change This PR aims to remove unused `getJobGraph` from `PipelineExecutorUtils`. ## Brief change log Remove unused `getJobGraph` from `PipelineExe

Re: [PR] [FLINK-37132][transform] validate meta schema in Multi Transform [flink-cdc]

2025-04-21 Thread via GitHub
MOBIN-F commented on PR #3865: URL: https://github.com/apache/flink-cdc/pull/3865#issuecomment-2819931254 > Thanks for your fix. But there is now a user scenario where users want to add fields of `__namespace_name__ || '.' || __schema_name__ || '.' || __table_name__ identifier_name` and `__

Re: [PR] [FLINK-35014] SqlNode to operation conversion for models [flink]

2025-04-21 Thread via GitHub
fsk119 commented on code in PR #25834: URL: https://github.com/apache/flink/pull/25834#discussion_r2053197415 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/operations/converters/SqlAlterModelSetConverter.java: ## @@ -0,0 +1,75 @@ +/* + * License

Re: [PR] [FLINK-35014] SqlNode to operation conversion for models [flink]

2025-04-21 Thread via GitHub
fsk119 commented on code in PR #25834: URL: https://github.com/apache/flink/pull/25834#discussion_r2053226463 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/operations/converters/SqlCreateModelConverter.java: ## @@ -0,0 +1,113 @@ +/* + * Licensed

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xuyangzhong commented on code in PR #26379: URL: https://github.com/apache/flink/pull/26379#discussion_r2053192854 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/reuse/SinkReuser.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Soft

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xuyangzhong merged PR #26379: URL: https://github.com/apache/flink/pull/26379 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.a

[jira] [Created] (FLINK-37710) Schema operator relies on data sources re-emitting CreateTableEvents during startup

2025-04-21 Thread Sergei Morozov (Jira)
Sergei Morozov created FLINK-37710: -- Summary: Schema operator relies on data sources re-emitting CreateTableEvents during startup Key: FLINK-37710 URL: https://issues.apache.org/jira/browse/FLINK-37710

Re: [PR] [FLINK-37697][runtime] Fix application mode cluster shut down before … [flink]

2025-04-21 Thread via GitHub
eemario commented on code in PR #26489: URL: https://github.com/apache/flink/pull/26489#discussion_r2053286838 ## flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/Dispatcher.java: ## @@ -1093,13 +1093,25 @@ public CompletableFuture stopWithSavepointAndGetLocation

Re: [PR] [FLINK-37509] Increment scan source no need to buffer data when enable skipping backfill. [flink-cdc]

2025-04-21 Thread via GitHub
loserwang1024 commented on code in PR #3964: URL: https://github.com/apache/flink-cdc/pull/3964#discussion_r2053397927 ## flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base/src/main/java/org/apache/flink/cdc/connectors/base/source/reader/external/IncrementalSourceScanFetcher.j

[jira] [Updated] (FLINK-37710) Schema operator relies on data sources re-emitting CreateTableEvents during startup

2025-04-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37710: --- Labels: pull-request-available (was: ) > Schema operator relies on data sources re-emitting

Re: [PR] [FLINK-37604] Generate static UIDs for pipeline operators [flink-cdc]

2025-04-21 Thread via GitHub
yuxiqian commented on PR #3977: URL: https://github.com/apache/flink-cdc/pull/3977#issuecomment-2819913015 Thanks for @morozov's quick response. Do you think we need an IT case to verify if operator UIDs are correctly set by examining Flink execution plan JSON (like #3887)? -- This is an

[jira] [Commented] (FLINK-37537) Duplicate key when job failover in SchemaOperator

2025-04-21 Thread Yanquan Lv (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17946313#comment-17946313 ] Yanquan Lv commented on FLINK-37537: I would like to take this.  > Duplicate key wh

Re: [PR] [FLINK-37705][rest] JobPlanInfo properties spec adjustments [flink]

2025-04-21 Thread via GitHub
Efrat19 commented on PR #26488: URL: https://github.com/apache/flink/pull/26488#issuecomment-2820209268 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [FLINK-35093][postgres] Add COMMITTED_OFFSET startup mode for Postgres CDC to resume from existing replication slot [flink-cdc]

2025-04-21 Thread via GitHub
phamvinh1712 commented on code in PR #3950: URL: https://github.com/apache/flink-cdc/pull/3950#discussion_r2053449485 ## flink-cdc-connect/flink-cdc-source-connectors/flink-connector-postgres-cdc/src/test/java/org/apache/flink/cdc/connectors/postgres/table/PostgreSQLConnectorITCase.java

[jira] [Closed] (FLINK-37543) Support sink reuse in batch mode

2025-04-21 Thread Xuyang Zhong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuyang Zhong closed FLINK-37543. Fix Version/s: 2.1.0 Assignee: xiangyu feng Resolution: Fixed > Support sink reuse

[jira] [Commented] (FLINK-37543) Support sink reuse in batch mode

2025-04-21 Thread Xuyang Zhong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17946274#comment-17946274 ] Xuyang Zhong commented on FLINK-37543: -- Merge into master: c724168fad4215626b5596dd

[jira] [Updated] (FLINK-37711) Remove unused getJobGraph from PipelineExecutorUtils

2025-04-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37711: --- Labels: pull-request-available (was: ) > Remove unused getJobGraph from PipelineExecutorUti

Re: [PR] [FLINK-34874][cdc-connector][mongodb] Support initial.snapshotting.pipeline related configs in table api [flink-cdc]

2025-04-21 Thread via GitHub
leonardBang commented on PR #3707: URL: https://github.com/apache/flink-cdc/pull/3707#issuecomment-2819989172 @herunkang2018 Would you like to update this PR to address @yuxiqian 's comment? -- This is an automated message from the Apache Git Service. To respond to the message, pleas

Re: [PR] [FLINK-37509] Increment scan source no need to buffer data when enable skipping backfill. [flink-cdc]

2025-04-21 Thread via GitHub
yuxiqian commented on code in PR #3964: URL: https://github.com/apache/flink-cdc/pull/3964#discussion_r2053282655 ## flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base/src/main/java/org/apache/flink/cdc/connectors/base/source/reader/external/IncrementalSourceScanFetcher.java:

Re: [PR] [FLINK-37711] Remove unused getJobGraph from PipelineExecutorUtils [flink]

2025-04-21 Thread via GitHub
flinkbot commented on PR #26493: URL: https://github.com/apache/flink/pull/26493#issuecomment-2819991176 ## CI report: * eae6b270529295280b4c600c23318e851701c5d3 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-37542][table-planner] Support sink reuse in streaming mode [flink]

2025-04-21 Thread via GitHub
xiangyuf commented on PR #26380: URL: https://github.com/apache/flink/pull/26380#issuecomment-2820224197 @xuyangzhong @reswqa Kindly remind for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[jira] [Resolved] (FLINK-36931) FlinkCDC YAML supports synchronizing the full amount of data of the entire database in Batch mode

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-36931. Fix Version/s: cdc-3.4.0 Resolution: Implemented Supported via master: 61702a6a9d729b1b

Re: [PR] [FLINK-36931][cdc] FlinkCDC YAML supports batch mode [flink-cdc]

2025-04-21 Thread via GitHub
leonardBang merged PR #3812: URL: https://github.com/apache/flink-cdc/pull/3812 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

[jira] [Assigned] (FLINK-37709) Add file descriptor usage in default JVM metrics

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-37709: -- Assignee: Zhenqiu Huang > Add file descriptor usage in default JVM metrics > ---

Re: [PR] [FLINK-36546] Handle batch sources in DataSinkTranslator [flink-cdc]

2025-04-21 Thread via GitHub
lvyanquan commented on PR #3646: URL: https://github.com/apache/flink-cdc/pull/3646#issuecomment-2820146856 Hi @morozov. You can rebase to master and test it with existed case in batch mode refer to https://github.com/apache/flink-cdc/pull/3812. -- This is an automated message from the Ap

Re: [PR] [FLINK-37509] Increment scan source no need to buffer data when enable skipping backfill. [flink-cdc]

2025-04-21 Thread via GitHub
loserwang1024 commented on code in PR #3964: URL: https://github.com/apache/flink-cdc/pull/3964#discussion_r2053383609 ## flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base/src/main/java/org/apache/flink/cdc/connectors/base/source/reader/external/IncrementalSourceScanFetcher.j

Re: [PR] [FLINK-37509] Increment scan source no need to buffer data when enable skipping backfill. [flink-cdc]

2025-04-21 Thread via GitHub
loserwang1024 commented on code in PR #3964: URL: https://github.com/apache/flink-cdc/pull/3964#discussion_r2053383609 ## flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base/src/main/java/org/apache/flink/cdc/connectors/base/source/reader/external/IncrementalSourceScanFetcher.j

Re: [PR] [FLINK-37509] Increment scan source no need to buffer data when enable skipping backfill. [flink-cdc]

2025-04-21 Thread via GitHub
loserwang1024 commented on code in PR #3964: URL: https://github.com/apache/flink-cdc/pull/3964#discussion_r2053383609 ## flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base/src/main/java/org/apache/flink/cdc/connectors/base/source/reader/external/IncrementalSourceScanFetcher.j

Re: [PR] [FLINK-37509] Increment scan source no need to buffer data when enable skipping backfill. [flink-cdc]

2025-04-21 Thread via GitHub
yuxiqian commented on code in PR #3964: URL: https://github.com/apache/flink-cdc/pull/3964#discussion_r2053389206 ## flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base/src/main/java/org/apache/flink/cdc/connectors/base/source/reader/external/IncrementalSourceScanFetcher.java:

[jira] [Assigned] (FLINK-37712) Met TableNotFound exception when reading ParallelMetadataSource to Paimon Sink

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-37712: -- Assignee: Yanquan Lv > Met TableNotFound exception when reading ParallelMetadataSource to Pai

[jira] [Created] (FLINK-37711) Remove unused getJobGraph from PipelineExecutorUtils

2025-04-21 Thread Jiaan Geng (Jira)
Jiaan Geng created FLINK-37711: -- Summary: Remove unused getJobGraph from PipelineExecutorUtils Key: FLINK-37711 URL: https://issues.apache.org/jira/browse/FLINK-37711 Project: Flink Issue Type:

[jira] [Commented] (FLINK-37684) ClassCastException: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.flink.streaming.runtime.partitioner.KeyGroupStreamPartitioner.keySe

2025-04-21 Thread Ferenc Csaky (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17946292#comment-17946292 ] Ferenc Csaky commented on FLINK-37684: -- Is this still reproducible in Flink 1.20? T

Re: [PR] [FLINK-31275] extend lineage listener with QueryOperationEvent [flink]

2025-04-21 Thread via GitHub
HuangZhenQiu commented on PR #26089: URL: https://github.com/apache/flink/pull/26089#issuecomment-2819016883 @pawel-big-lebowski Is the CI still failing? Would you please resolve it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] [FLINK-29549]- Flink Glue Catalog integration [flink-connector-aws]

2025-04-21 Thread via GitHub
FranMorilloAWS commented on code in PR #191: URL: https://github.com/apache/flink-connector-aws/pull/191#discussion_r2052176250 ## flink-catalog-aws-glue/src/main/java/org/apache/flink/table/catalog/glue/operations/GlueDatabaseOperations.java: ## @@ -0,0 +1,240 @@ +/* + * Licen

Re: [PR] [FLINK-37507] Fix MySQL CDC accidentally captures common-prefix database when `scan.binlog.newly-added-table` is enabled [flink-cdc]

2025-04-21 Thread via GitHub
leonardBang merged PR #3957: URL: https://github.com/apache/flink-cdc/pull/3957 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

[jira] [Assigned] (FLINK-37507) MySQL CDC captures common-prefix database accidentally when scan.binlog.newly-added-table option is enabled

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-37507: -- Assignee: yux > MySQL CDC captures common-prefix database accidentally when > scan.binlog.ne

Re: [PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

2025-04-21 Thread via GitHub
ChaomingZhangCN commented on PR #3887: URL: https://github.com/apache/flink-cdc/pull/3887#issuecomment-2818044280 ping @leonardBang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

2025-04-21 Thread via GitHub
leonardBang commented on PR #3887: URL: https://github.com/apache/flink-cdc/pull/3887#issuecomment-2818053249 @ChaomingZhangCN Could you rebase this PR to latest master branch as it bumped junit version to junit5 for all tests? -- This is an automated message from the Apache Git Service.

Re: [PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

2025-04-21 Thread via GitHub
leonardBang commented on code in PR #3887: URL: https://github.com/apache/flink-cdc/pull/3887#discussion_r2052201271 ## flink-cdc-cli/src/main/java/org/apache/flink/cdc/cli/parser/YamlPipelineDefinitionParser.java: ## @@ -188,7 +189,14 @@ private SourceDef toSourceDef(JsonNode s

Re: [PR] [FLINK-37542][table-planner] Support sink reuse in streaming mode [flink]

2025-04-21 Thread via GitHub
xiangyuf commented on PR #26380: URL: https://github.com/apache/flink/pull/26380#issuecomment-2818179237 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] Handle dwh delete op [flink-connector-jdbc]

2025-04-21 Thread via GitHub
hieu2102 opened a new pull request, #163: URL: https://github.com/apache/flink-connector-jdbc/pull/163 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] [FLINK-37509] Increment scan source no need to buffer data when enable skipping backfill. [flink-cdc]

2025-04-21 Thread via GitHub
leonardBang commented on PR #3964: URL: https://github.com/apache/flink-cdc/pull/3964#issuecomment-2817985904 @loserwang1024 Would you like to rebase to latest master branch for review now? -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

2025-04-21 Thread via GitHub
yuxiqian commented on PR #3887: URL: https://github.com/apache/flink-cdc/pull/3887#issuecomment-2818105857 Minor suggestion: shall we pick `MySqlToPaimonE2eITCase` and `FlinkParallelizedPipelineITCase` to be executed with different source / sink parallelism? I believe they could cover most

[jira] [Commented] (FLINK-37706) [Flink CDC] Oceanbase OceanBaseMetadataApplierTest failed due to Caused by: java.sql.SQLException: Server out of disk space

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17946108#comment-17946108 ] Leonard Xu commented on FLINK-37706: We used latest OB docker version in test, but r

Re: [PR] [FLINK-37705][rest] JobPlanInfo properties spec adjustments [flink]

2025-04-21 Thread via GitHub
Efrat19 commented on PR #26488: URL: https://github.com/apache/flink/pull/26488#issuecomment-2818475344 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] [FLINK-37688] Implement Amazon CloudWatch Metric Sink Connector [flink-connector-aws]

2025-04-21 Thread via GitHub
darenwkt opened a new pull request, #202: URL: https://github.com/apache/flink-connector-aws/pull/202 ## Purpose of the change Implement Amazon CloudWatch Metric Sink Connector which includes following: - DataStream and TableAPI support - Integration Test with LocalStack

[jira] [Updated] (FLINK-37688) FLIP-524: Amazon CloudWatch Metric Sink Connector

2025-04-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37688: --- Labels: pull-request-available (was: ) > FLIP-524: Amazon CloudWatch Metric Sink Connector

Re: [PR] [FLINK-29549]- Flink Glue Catalog integration [flink-connector-aws]

2025-04-21 Thread via GitHub
FranMorilloAWS commented on code in PR #191: URL: https://github.com/apache/flink-connector-aws/pull/191#discussion_r2052171687 ## flink-catalog-aws-glue/src/main/java/org/apache/flink/table/catalog/glue/util/GlueTypeConverter.java: ## @@ -0,0 +1,311 @@ +/* + * Licensed to the

Re: [PR] [FLINK-37132][transform] validate meta schema in Multi Transform [flink-cdc]

2025-04-21 Thread via GitHub
lvyanquan commented on code in PR #3865: URL: https://github.com/apache/flink-cdc/pull/3865#discussion_r2052368481 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/operators/transform/PreTransformOperator.java: ## @@ -411,18 +411,16 @@ private void cacheTransformR

[jira] [Commented] (FLINK-37673) The transformation of table metadata should only be effective for the first matching rule

2025-04-21 Thread Yanquan Lv (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17946125#comment-17946125 ] Yanquan Lv commented on FLINK-37673: Although using the first rule is more intuitive

[jira] [Commented] (FLINK-37226) Replace deprecated methods in the DataStream API introduction doc

2025-04-21 Thread Zhanghao Chen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17946128#comment-17946128 ] Zhanghao Chen commented on FLINK-37226: --- [~leejeonghwan] Sure, go ahead > Replace

Re: [PR] [FLINK-37697][runtime] Fix application mode cluster shut down before … [flink]

2025-04-21 Thread via GitHub
JunRuiLee commented on code in PR #26489: URL: https://github.com/apache/flink/pull/26489#discussion_r2052656518 ## flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/Dispatcher.java: ## @@ -1596,6 +1608,12 @@ CompletableFuture getJobTerminationFuture(JobID jobId)

Re: [PR] [FLINK-37697][runtime] Fix application mode cluster shut down before … [flink]

2025-04-21 Thread via GitHub
JunRuiLee commented on code in PR #26489: URL: https://github.com/apache/flink/pull/26489#discussion_r2052653921 ## flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/Dispatcher.java: ## @@ -1093,13 +1093,25 @@ public CompletableFuture stopWithSavepointAndGetLocati

Re: [PR] [FLINK-37132][transform] validate meta schema in Multi Transform [flink-cdc]

2025-04-21 Thread via GitHub
MOBIN-F commented on PR #3865: URL: https://github.com/apache/flink-cdc/pull/3865#issuecomment-2818133148 @yuxiqian @lvyanquan PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[jira] [Assigned] (FLINK-37509) Increment scan source no need to buffer data when enable skipping backfill.

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-37509: -- Assignee: Hongshun Wang > Increment scan source no need to buffer data when enable skipping b

[jira] [Resolved] (FLINK-37507) MySQL CDC captures common-prefix database accidentally when scan.binlog.newly-added-table option is enabled

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-37507. Fix Version/s: cdc-3.4.0 Resolution: Fixed fixed via master: 72345f0ec73c9cc3013d2804c90202

Re: [PR] [FLINK-29549]- Flink Glue Catalog integration [flink-connector-aws]

2025-04-21 Thread via GitHub
FranMorilloAWS commented on code in PR #191: URL: https://github.com/apache/flink-connector-aws/pull/191#discussion_r2052228672 ## flink-catalog-aws-glue/src/main/java/org/apache/flink/table/catalog/glue/GlueCatalog.java: ## @@ -0,0 +1,925 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Handle dwh delete op [flink-connector-jdbc]

2025-04-21 Thread via GitHub
hieu2102 closed pull request #163: Handle dwh delete op URL: https://github.com/apache/flink-connector-jdbc/pull/163 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] Handle dwh delete op [flink-connector-jdbc]

2025-04-21 Thread via GitHub
boring-cyborg[bot] commented on PR #163: URL: https://github.com/apache/flink-connector-jdbc/pull/163#issuecomment-2818188413 Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html) -- This is

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xuyangzhong commented on code in PR #26379: URL: https://github.com/apache/flink/pull/26379#discussion_r2052275668 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/reuse/SinkReuser.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Soft

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xuyangzhong commented on code in PR #26379: URL: https://github.com/apache/flink/pull/26379#discussion_r2052274380 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/reuse/SinkReuser.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Soft

[jira] [Updated] (FLINK-37708) Print digest about SinkAbilitySpec in plan just like SourceAbilitySpec

2025-04-21 Thread Xuyang Zhong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuyang Zhong updated FLINK-37708: - Description: [https://github.com/apache/flink/pull/26379#discussion_r2048105086]   Maybe we shou

[jira] [Updated] (FLINK-37708) Print digest about SinkAbilitySpec in plan just like SourceAbilitySpec

2025-04-21 Thread Xuyang Zhong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuyang Zhong updated FLINK-37708: - Affects Version/s: 1.20.1 1.19.2 2.0.0 > Print dig

[jira] [Updated] (FLINK-37708) Print digest about SinkAbilitySpec in plan just like SourceAbilitySpec

2025-04-21 Thread Xuyang Zhong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuyang Zhong updated FLINK-37708: - Component/s: Table SQL / Planner > Print digest about SinkAbilitySpec in plan just like SourceAb

Re: [PR] [FLINK-37132][transform] validate meta schema in Multi Transform [flink-cdc]

2025-04-21 Thread via GitHub
lvyanquan commented on PR #3865: URL: https://github.com/apache/flink-cdc/pull/3865#issuecomment-2818203231 Thanks for your fix. But there is now a user scenario where users want to add fields of `__namespace_name__ || '.' || __schema_name__ || '.' || __table_name__ identifier_name` and

[jira] [Created] (FLINK-37708) Print digest about SinkAbilitySpec in plan just like SourceAbilitySpec

2025-04-21 Thread Xuyang Zhong (Jira)
Xuyang Zhong created FLINK-37708: Summary: Print digest about SinkAbilitySpec in plan just like SourceAbilitySpec Key: FLINK-37708 URL: https://issues.apache.org/jira/browse/FLINK-37708 Project: Flink

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xuyangzhong commented on PR #26379: URL: https://github.com/apache/flink/pull/26379#issuecomment-2818218602 @xiangyuf Hi, thanks for updating. I just some other minor comments. BTW, @davidradl do you have more thoughts about this pr? -- This is an automated message from the Apache Git Ser

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xuyangzhong commented on code in PR #26379: URL: https://github.com/apache/flink/pull/26379#discussion_r2052281571 ## flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/batch/sql/BatchSinkReuseTest.xml: ## @@ -0,0 +1,243 @@ + + + + +

[PR] [FLINK-37697][runtime] Fix application mode cluster shut down before … [flink]

2025-04-21 Thread via GitHub
eemario opened a new pull request, #26489: URL: https://github.com/apache/flink/pull/26489 …all job termination futures complete ## What is the purpose of the change *This pull request makes the cluster shutdown after all job termination futures complete in applicat

Re: [PR] [FLINK-37705][rest] JobPlanInfo properties spec adjustments [flink]

2025-04-21 Thread via GitHub
Efrat19 commented on PR #26488: URL: https://github.com/apache/flink/pull/26488#issuecomment-2818239895 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[jira] [Updated] (FLINK-37697) ExecutionGraphInfo may not be archived when cluster shutdown in application mode

2025-04-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37697: --- Labels: pull-request-available (was: ) > ExecutionGraphInfo may not be archived when cluste

Re: [PR] [FLINK-29549]- Flink Glue Catalog integration [flink-connector-aws]

2025-04-21 Thread via GitHub
FranMorilloAWS commented on code in PR #191: URL: https://github.com/apache/flink-connector-aws/pull/191#discussion_r2052295570 ## flink-catalog-aws-glue/src/main/java/org/apache/flink/table/catalog/glue/operations/GlueTableOperations.java: ## @@ -0,0 +1,255 @@ +/* + * Licensed

[jira] [Assigned] (FLINK-37706) [Flink CDC] Oceanbase OceanBaseMetadataApplierTest failed due to Caused by: java.sql.SQLException: Server out of disk space

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-37706: -- Assignee: Leonard Xu > [Flink CDC] Oceanbase OceanBaseMetadataApplierTest failed due to Cause

  1   2   >