Re: [PR] [FLINK-35093][postgres] Add COMMITTED_OFFSET startup mode for Postgres CDC to resume from existing replication slot [flink-cdc]

2025-04-21 Thread via GitHub
phamvinh1712 commented on code in PR #3950: URL: https://github.com/apache/flink-cdc/pull/3950#discussion_r2053449485 ## flink-cdc-connect/flink-cdc-source-connectors/flink-connector-postgres-cdc/src/test/java/org/apache/flink/cdc/connectors/postgres/table/PostgreSQLConnectorITCase.java

Re: [PR] [FLINK-37542][table-planner] Support sink reuse in streaming mode [flink]

2025-04-21 Thread via GitHub
xiangyuf commented on PR #26380: URL: https://github.com/apache/flink/pull/26380#issuecomment-2820224197 @xuyangzhong @reswqa Kindly remind for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] [FLINK-37705][rest] JobPlanInfo properties spec adjustments [flink]

2025-04-21 Thread via GitHub
Efrat19 commented on PR #26488: URL: https://github.com/apache/flink/pull/26488#issuecomment-2820209268 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[jira] [Commented] (FLINK-37537) Duplicate key when job failover in SchemaOperator

2025-04-21 Thread Yanquan Lv (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17946313#comment-17946313 ] Yanquan Lv commented on FLINK-37537: I would like to take this.  > Duplicate key wh

Re: [PR] [FLINK-37509] Increment scan source no need to buffer data when enable skipping backfill. [flink-cdc]

2025-04-21 Thread via GitHub
loserwang1024 commented on code in PR #3964: URL: https://github.com/apache/flink-cdc/pull/3964#discussion_r2053397927 ## flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base/src/main/java/org/apache/flink/cdc/connectors/base/source/reader/external/IncrementalSourceScanFetcher.j

[jira] [Assigned] (FLINK-37712) Met TableNotFound exception when reading ParallelMetadataSource to Paimon Sink

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-37712: -- Assignee: Yanquan Lv > Met TableNotFound exception when reading ParallelMetadataSource to Pai

Re: [PR] [FLINK-37509] Increment scan source no need to buffer data when enable skipping backfill. [flink-cdc]

2025-04-21 Thread via GitHub
yuxiqian commented on code in PR #3964: URL: https://github.com/apache/flink-cdc/pull/3964#discussion_r2053389206 ## flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base/src/main/java/org/apache/flink/cdc/connectors/base/source/reader/external/IncrementalSourceScanFetcher.java:

Re: [PR] [FLINK-37509] Increment scan source no need to buffer data when enable skipping backfill. [flink-cdc]

2025-04-21 Thread via GitHub
loserwang1024 commented on code in PR #3964: URL: https://github.com/apache/flink-cdc/pull/3964#discussion_r2053383609 ## flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base/src/main/java/org/apache/flink/cdc/connectors/base/source/reader/external/IncrementalSourceScanFetcher.j

Re: [PR] [FLINK-37509] Increment scan source no need to buffer data when enable skipping backfill. [flink-cdc]

2025-04-21 Thread via GitHub
loserwang1024 commented on code in PR #3964: URL: https://github.com/apache/flink-cdc/pull/3964#discussion_r2053383609 ## flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base/src/main/java/org/apache/flink/cdc/connectors/base/source/reader/external/IncrementalSourceScanFetcher.j

Re: [PR] [FLINK-37509] Increment scan source no need to buffer data when enable skipping backfill. [flink-cdc]

2025-04-21 Thread via GitHub
loserwang1024 commented on code in PR #3964: URL: https://github.com/apache/flink-cdc/pull/3964#discussion_r2053383609 ## flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base/src/main/java/org/apache/flink/cdc/connectors/base/source/reader/external/IncrementalSourceScanFetcher.j

Re: [PR] [FLINK-36546] Handle batch sources in DataSinkTranslator [flink-cdc]

2025-04-21 Thread via GitHub
lvyanquan commented on PR #3646: URL: https://github.com/apache/flink-cdc/pull/3646#issuecomment-2820146856 Hi @morozov. You can rebase to master and test it with existed case in batch mode refer to https://github.com/apache/flink-cdc/pull/3812. -- This is an automated message from the Ap

[jira] [Assigned] (FLINK-37709) Add file descriptor usage in default JVM metrics

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-37709: -- Assignee: Zhenqiu Huang > Add file descriptor usage in default JVM metrics > ---

[jira] [Resolved] (FLINK-36931) FlinkCDC YAML supports synchronizing the full amount of data of the entire database in Batch mode

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-36931. Fix Version/s: cdc-3.4.0 Resolution: Implemented Supported via master: 61702a6a9d729b1b

Re: [PR] [FLINK-36931][cdc] FlinkCDC YAML supports batch mode [flink-cdc]

2025-04-21 Thread via GitHub
leonardBang merged PR #3812: URL: https://github.com/apache/flink-cdc/pull/3812 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

[jira] [Commented] (FLINK-37684) ClassCastException: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.flink.streaming.runtime.partitioner.KeyGroupStreamPartitioner.keySe

2025-04-21 Thread Ferenc Csaky (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17946292#comment-17946292 ] Ferenc Csaky commented on FLINK-37684: -- Is this still reproducible in Flink 1.20? T

Re: [PR] [hotfix] [base] Fix the serialization error occurring with table names containing dot [flink-cdc]

2025-04-21 Thread via GitHub
leonardBang commented on code in PR #2705: URL: https://github.com/apache/flink-cdc/pull/2705#discussion_r2053300902 ## flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base/src/test/java/org/apache/flink/cdc/connectors/base/source/meta/split/SourceSplitSerializerTest.java: #

Re: [PR] [hotfix] Rename SessionDispatcherFactory to DefaultDispatcherFactory [flink]

2025-04-21 Thread via GitHub
beliefer commented on PR #26490: URL: https://github.com/apache/flink/pull/26490#issuecomment-2820015112 ping @tillrohrmann @zentol @kl0u @XComp @JunRuiLee cc @1996fanrui @davidradl @GOODBOY008 -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Updated] (FLINK-37712) Met TableNotFound exception when reading ParallelMetadataSource to Paimon Sink

2025-04-21 Thread Yanquan Lv (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanquan Lv updated FLINK-37712: --- Description: when using ParallelMetadataSource as Source, the Flink topology will be like: ---> 1.

[jira] [Updated] (FLINK-37712) Met TableNotFound exception when reading ParallelMetadataSource to Paimon Sink

2025-04-21 Thread Yanquan Lv (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanquan Lv updated FLINK-37712: --- Component/s: Flink CDC Affects Version/s: cdc-3.4.0 Description: when using Para

Re: [PR] [FLINK-37697][runtime] Fix application mode cluster shut down before … [flink]

2025-04-21 Thread via GitHub
JunRuiLee commented on code in PR #26489: URL: https://github.com/apache/flink/pull/26489#discussion_r2053295509 ## flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/Dispatcher.java: ## @@ -1256,7 +1268,8 @@ private void setClientHeartbeatTimeoutForInitializedJob()

[jira] [Updated] (FLINK-37712) Met TableNotFound exception when reading ParallelMetadataSource to Paimon Sink

2025-04-21 Thread Yanquan Lv (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanquan Lv updated FLINK-37712: --- Summary: Met TableNotFound exception when reading ParallelMetadataSource to Paimon Sink (was: Table

[jira] [Updated] (FLINK-37712) TableNotFound Exception when reading ParallelMetadataSource to Paimon Sink

2025-04-21 Thread Yanquan Lv (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanquan Lv updated FLINK-37712: --- Summary: TableNotFound Exception when reading ParallelMetadataSource to Paimon Sink (was: Table) >

Re: [PR] [FLINK-35093][postgres] Add COMMITTED_OFFSET startup mode for Postgres CDC to resume from existing replication slot [flink-cdc]

2025-04-21 Thread via GitHub
loserwang1024 commented on code in PR #3950: URL: https://github.com/apache/flink-cdc/pull/3950#discussion_r2053292573 ## flink-cdc-connect/flink-cdc-source-connectors/flink-connector-postgres-cdc/src/test/java/org/apache/flink/cdc/connectors/postgres/table/PostgreSQLConnectorITCase.jav

[jira] [Created] (FLINK-37712) Table

2025-04-21 Thread Yanquan Lv (Jira)
Yanquan Lv created FLINK-37712: -- Summary: Table Key: FLINK-37712 URL: https://issues.apache.org/jira/browse/FLINK-37712 Project: Flink Issue Type: Bug Reporter: Yanquan Lv -- T

Re: [PR] [FLINK-37697][runtime] Fix application mode cluster shut down before … [flink]

2025-04-21 Thread via GitHub
eemario commented on code in PR #26489: URL: https://github.com/apache/flink/pull/26489#discussion_r2053286838 ## flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/Dispatcher.java: ## @@ -1093,13 +1093,25 @@ public CompletableFuture stopWithSavepointAndGetLocation

Re: [PR] [FLINK-37711] Remove unused getJobGraph from PipelineExecutorUtils [flink]

2025-04-21 Thread via GitHub
flinkbot commented on PR #26493: URL: https://github.com/apache/flink/pull/26493#issuecomment-2819991176 ## CI report: * eae6b270529295280b4c600c23318e851701c5d3 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-37509] Increment scan source no need to buffer data when enable skipping backfill. [flink-cdc]

2025-04-21 Thread via GitHub
yuxiqian commented on code in PR #3964: URL: https://github.com/apache/flink-cdc/pull/3964#discussion_r2053282655 ## flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base/src/main/java/org/apache/flink/cdc/connectors/base/source/reader/external/IncrementalSourceScanFetcher.java:

Re: [PR] [FLINK-34874][cdc-connector][mongodb] Support initial.snapshotting.pipeline related configs in table api [flink-cdc]

2025-04-21 Thread via GitHub
leonardBang commented on PR #3707: URL: https://github.com/apache/flink-cdc/pull/3707#issuecomment-2819989172 @herunkang2018 Would you like to update this PR to address @yuxiqian 's comment? -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[jira] [Updated] (FLINK-37711) Remove unused getJobGraph from PipelineExecutorUtils

2025-04-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37711: --- Labels: pull-request-available (was: ) > Remove unused getJobGraph from PipelineExecutorUti

[PR] [FLINK-37711] Remove unused getJobGraph from PipelineExecutorUtils [flink]

2025-04-21 Thread via GitHub
beliefer opened a new pull request, #26493: URL: https://github.com/apache/flink/pull/26493 ## What is the purpose of the change This PR aims to remove unused `getJobGraph` from `PipelineExecutorUtils`. ## Brief change log Remove unused `getJobGraph` from `PipelineExe

[jira] [Created] (FLINK-37711) Remove unused getJobGraph from PipelineExecutorUtils

2025-04-21 Thread Jiaan Geng (Jira)
Jiaan Geng created FLINK-37711: -- Summary: Remove unused getJobGraph from PipelineExecutorUtils Key: FLINK-37711 URL: https://issues.apache.org/jira/browse/FLINK-37711 Project: Flink Issue Type:

Re: [PR] [FLINK-37132][transform] validate meta schema in Multi Transform [flink-cdc]

2025-04-21 Thread via GitHub
MOBIN-F commented on PR #3865: URL: https://github.com/apache/flink-cdc/pull/3865#issuecomment-2819931254 > Thanks for your fix. But there is now a user scenario where users want to add fields of `__namespace_name__ || '.' || __schema_name__ || '.' || __table_name__ identifier_name` and `__

Re: [PR] [FLINK-37604] Generate static UIDs for pipeline operators [flink-cdc]

2025-04-21 Thread via GitHub
yuxiqian commented on PR #3977: URL: https://github.com/apache/flink-cdc/pull/3977#issuecomment-2819913015 Thanks for @morozov's quick response. Do you think we need an IT case to verify if operator UIDs are correctly set by examining Flink execution plan JSON (like #3887)? -- This is an

Re: [PR] [FLINK-35014] SqlNode to operation conversion for models [flink]

2025-04-21 Thread via GitHub
fsk119 commented on code in PR #25834: URL: https://github.com/apache/flink/pull/25834#discussion_r2053226463 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/operations/converters/SqlCreateModelConverter.java: ## @@ -0,0 +1,113 @@ +/* + * Licensed

Re: [PR] [FLINK-35014] SqlNode to operation conversion for models [flink]

2025-04-21 Thread via GitHub
fsk119 commented on code in PR #25834: URL: https://github.com/apache/flink/pull/25834#discussion_r2053197415 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/operations/converters/SqlAlterModelSetConverter.java: ## @@ -0,0 +1,75 @@ +/* + * License

[jira] [Updated] (FLINK-37710) Schema operator relies on data sources re-emitting CreateTableEvents during startup

2025-04-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37710: --- Labels: pull-request-available (was: ) > Schema operator relies on data sources re-emitting

[jira] [Closed] (FLINK-37543) Support sink reuse in batch mode

2025-04-21 Thread Xuyang Zhong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuyang Zhong closed FLINK-37543. Fix Version/s: 2.1.0 Assignee: xiangyu feng Resolution: Fixed > Support sink reuse

[jira] [Commented] (FLINK-37543) Support sink reuse in batch mode

2025-04-21 Thread Xuyang Zhong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17946274#comment-17946274 ] Xuyang Zhong commented on FLINK-37543: -- Merge into master: c724168fad4215626b5596dd

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xuyangzhong merged PR #26379: URL: https://github.com/apache/flink/pull/26379 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.a

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xuyangzhong commented on code in PR #26379: URL: https://github.com/apache/flink/pull/26379#discussion_r2053192854 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/reuse/SinkReuser.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Soft

[jira] [Created] (FLINK-37710) Schema operator relies on data sources re-emitting CreateTableEvents during startup

2025-04-21 Thread Sergei Morozov (Jira)
Sergei Morozov created FLINK-37710: -- Summary: Schema operator relies on data sources re-emitting CreateTableEvents during startup Key: FLINK-37710 URL: https://issues.apache.org/jira/browse/FLINK-37710

[jira] [Resolved] (FLINK-37706) [Flink CDC] Oceanbase OceanBaseMetadataApplierTest failed due to Caused by: java.sql.SQLException: Server out of disk space

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-37706. Resolution: Fixed via master: 4cf54c61f9f9283ba1138ffc0179f53f6b67ce3f > [Flink CDC] Oceanbase Oc

Re: [PR] [FLINK-37706][source-connector/oceanbase][tests] Use lts docker image version instead of latest for test [flink-cdc]

2025-04-21 Thread via GitHub
leonardBang merged PR #3998: URL: https://github.com/apache/flink-cdc/pull/3998 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [FLINK-37709][metrics] add file descriptor jvm metrics [flink]

2025-04-21 Thread via GitHub
flinkbot commented on PR #26492: URL: https://github.com/apache/flink/pull/26492#issuecomment-2819731193 ## CI report: * 6fae8ca00d068e5e2564a572eb95339d356c43d4 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Updated] (FLINK-37709) Add file descriptor usage in default JVM metrics

2025-04-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37709: --- Labels: pull-request-available (was: ) > Add file descriptor usage in default JVM metrics

[PR] [FLINK-37709][metrics] add file descriptor jvm metrics [flink]

2025-04-21 Thread via GitHub
HuangZhenQiu opened a new pull request, #26492: URL: https://github.com/apache/flink/pull/26492 ## What is the purpose of the change Add file descriptor related JVM metrics ## Brief change log - Add the default registration in MetricUtils - Added test in MetricUtilsTes

[jira] [Commented] (FLINK-37226) Replace deprecated methods in the DataStream API introduction doc

2025-04-21 Thread JH LEE (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17946247#comment-17946247 ] JH LEE commented on FLINK-37226: [~Zhanghao Chen] Should I modify only the documentatio

[jira] [Created] (FLINK-37709) Add file descriptor usage in default JVM metrics

2025-04-21 Thread Zhenqiu Huang (Jira)
Zhenqiu Huang created FLINK-37709: - Summary: Add file descriptor usage in default JVM metrics Key: FLINK-37709 URL: https://issues.apache.org/jira/browse/FLINK-37709 Project: Flink Issue Typ

Re: [PR] [FLINK-35093][postgres] Add COMMITTED_OFFSET startup mode for Postgres CDC to resume from existing replication slot [flink-cdc]

2025-04-21 Thread via GitHub
phamvinh1712 commented on code in PR #3950: URL: https://github.com/apache/flink-cdc/pull/3950#discussion_r2053051726 ## flink-cdc-connect/flink-cdc-source-connectors/flink-connector-postgres-cdc/src/test/java/org/apache/flink/cdc/connectors/postgres/table/PostgreSQLConnectorITCase.java

Re: [PR] [FLINK-35093][postgres] Add COMMITTED_OFFSET startup mode for Postgres CDC to resume from existing replication slot [flink-cdc]

2025-04-21 Thread via GitHub
phamvinh1712 commented on code in PR #3950: URL: https://github.com/apache/flink-cdc/pull/3950#discussion_r2053052981 ## flink-cdc-connect/flink-cdc-source-connectors/flink-connector-postgres-cdc/src/main/java/io/debezium/connector/postgresql/connection/PostgresConnectionUtils.java: ###

Re: [PR] [FLINK-35093][postgres] Add COMMITTED_OFFSET startup mode for Postgres CDC to resume from existing replication slot [flink-cdc]

2025-04-21 Thread via GitHub
phamvinh1712 commented on code in PR #3950: URL: https://github.com/apache/flink-cdc/pull/3950#discussion_r2053051726 ## flink-cdc-connect/flink-cdc-source-connectors/flink-connector-postgres-cdc/src/test/java/org/apache/flink/cdc/connectors/postgres/table/PostgreSQLConnectorITCase.java

[jira] [Updated] (FLINK-37688) FLIP-524: Amazon CloudWatch Metric Sink Connector

2025-04-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37688: --- Labels: pull-request-available (was: ) > FLIP-524: Amazon CloudWatch Metric Sink Connector

[PR] [FLINK-37688] Implement Amazon CloudWatch Metric Sink Connector [flink-connector-aws]

2025-04-21 Thread via GitHub
darenwkt opened a new pull request, #202: URL: https://github.com/apache/flink-connector-aws/pull/202 ## Purpose of the change Implement Amazon CloudWatch Metric Sink Connector which includes following: - DataStream and TableAPI support - Integration Test with LocalStack

Re: [PR] [FLINK-36902] Remove support for FlinkSecurityManager [flink]

2025-04-21 Thread via GitHub
flinkbot commented on PR #26491: URL: https://github.com/apache/flink/pull/26491#issuecomment-2819343068 ## CI report: * 0fd0ed501b096ac5f7f83b18aadacdaf2623470d UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Updated] (FLINK-36902) Security Manager deprecation

2025-04-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-36902: --- Labels: pull-request-available (was: ) > Security Manager deprecation > ---

[PR] [FLINK-36902] Remove support for FlinkSecurityManager [flink]

2025-04-21 Thread via GitHub
cushon opened a new pull request, #26491: URL: https://github.com/apache/flink/pull/26491 ## What is the purpose of the change This change removes references to the Security Manager API, for compatibility with JDK 24 and newer versions. Support for the Security Manager [permanently d

Re: [PR] [hotfix][ci][docker] Use correct java version in snapshots workflow [flink-docker]

2025-04-21 Thread via GitHub
morazow commented on PR #223: URL: https://github.com/apache/flink-docker/pull/223#issuecomment-2819147070 Hey @dstreitfeld-sn, thanks, yes, yesterday we also observed the failure. I think it is related to the GitHub workflow matrix, I am going to look into it again 👍 -- This is an a

Re: [PR] [FLINK-31275] extend lineage listener with QueryOperationEvent [flink]

2025-04-21 Thread via GitHub
HuangZhenQiu commented on PR #26089: URL: https://github.com/apache/flink/pull/26089#issuecomment-2819016883 @pawel-big-lebowski Is the CI still failing? Would you please resolve it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] [hotfix][ci][docker] Use correct java version in snapshots workflow [flink-docker]

2025-04-21 Thread via GitHub
dstreitfeld-sn commented on PR #223: URL: https://github.com/apache/flink-docker/pull/223#issuecomment-2818936070 @XComp @morazow This did resolve that particular pipeline error (thanks!), but it looks like there's still [another issue](https://github.com/apache/flink-docker/actions) causin

Re: [PR] [FLINK-37697][runtime] Fix application mode cluster shut down before … [flink]

2025-04-21 Thread via GitHub
JunRuiLee commented on code in PR #26489: URL: https://github.com/apache/flink/pull/26489#discussion_r2052653921 ## flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/Dispatcher.java: ## @@ -1093,13 +1093,25 @@ public CompletableFuture stopWithSavepointAndGetLocati

Re: [PR] [FLINK-37697][runtime] Fix application mode cluster shut down before … [flink]

2025-04-21 Thread via GitHub
JunRuiLee commented on code in PR #26489: URL: https://github.com/apache/flink/pull/26489#discussion_r2052656518 ## flink-runtime/src/main/java/org/apache/flink/runtime/dispatcher/Dispatcher.java: ## @@ -1596,6 +1608,12 @@ CompletableFuture getJobTerminationFuture(JobID jobId)

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xiangyuf commented on PR #26379: URL: https://github.com/apache/flink/pull/26379#issuecomment-2818736336 > @xiangyuf Hi, thanks for updating. I just some other minor comments. BTW, @davidradl do you have more thoughts about this pr? @xuyangzhong Thx for review. Pls check my reply. -

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xiangyuf commented on code in PR #26379: URL: https://github.com/apache/flink/pull/26379#discussion_r2052567555 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/reuse/SinkReuser.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Softwar

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xiangyuf commented on code in PR #26379: URL: https://github.com/apache/flink/pull/26379#discussion_r2052542223 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/reuse/SinkReuser.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Softwar

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xiangyuf commented on code in PR #26379: URL: https://github.com/apache/flink/pull/26379#discussion_r2052498557 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/reuse/SinkReuser.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Softwar

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xiangyuf commented on code in PR #26379: URL: https://github.com/apache/flink/pull/26379#discussion_r2052502719 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/reuse/SinkReuser.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Softwar

Re: [PR] [FLINK-37705][rest] JobPlanInfo properties spec adjustments [flink]

2025-04-21 Thread via GitHub
Efrat19 commented on PR #26488: URL: https://github.com/apache/flink/pull/26488#issuecomment-2818475344 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[jira] [Commented] (FLINK-37226) Replace deprecated methods in the DataStream API introduction doc

2025-04-21 Thread Zhanghao Chen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17946128#comment-17946128 ] Zhanghao Chen commented on FLINK-37226: --- [~leejeonghwan] Sure, go ahead > Replace

[jira] [Commented] (FLINK-37673) The transformation of table metadata should only be effective for the first matching rule

2025-04-21 Thread Yanquan Lv (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17946125#comment-17946125 ] Yanquan Lv commented on FLINK-37673: Although using the first rule is more intuitive

Re: [PR] [FLINK-37132][transform] validate meta schema in Multi Transform [flink-cdc]

2025-04-21 Thread via GitHub
lvyanquan commented on code in PR #3865: URL: https://github.com/apache/flink-cdc/pull/3865#discussion_r2052368481 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/operators/transform/PreTransformOperator.java: ## @@ -411,18 +411,16 @@ private void cacheTransformR

Re: [PR] Add support for GaussDB database type [flink-connector-jdbc]

2025-04-21 Thread via GitHub
liubao68 commented on code in PR #162: URL: https://github.com/apache/flink-connector-jdbc/pull/162#discussion_r2052341651 ## flink-connector-jdbc-gaussdb/src/test/java/org/apache/flink/connector/jdbc/gaussdb/testutils/GaussDBContainer.java: ## @@ -0,0 +1,162 @@ +package org.ap

Re: [PR] [hotfix] Rename SessionDispatcherFactory to DefaultDispatcherFactory [flink]

2025-04-21 Thread via GitHub
flinkbot commented on PR #26490: URL: https://github.com/apache/flink/pull/26490#issuecomment-2818351193 ## CI report: * dc383c5d2a15a5287c3281f561188527ca11f2c1 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-29549]- Flink Glue Catalog integration [flink-connector-aws]

2025-04-21 Thread via GitHub
FranMorilloAWS commented on code in PR #191: URL: https://github.com/apache/flink-connector-aws/pull/191#discussion_r2052347317 ## flink-catalog-aws-glue/src/main/java/org/apache/flink/table/catalog/glue/GlueCatalog.java: ## @@ -0,0 +1,925 @@ +/* + * Licensed to the Apache Soft

[PR] [hotfix] Rename SessionDispatcherFactory to DefaultDispatcherFactory [flink]

2025-04-21 Thread via GitHub
beliefer opened a new pull request, #26490: URL: https://github.com/apache/flink/pull/26490 ## What is the purpose of the change This PR proposes to rename `SessionDispatcherFactory` to `DefaultDispatcherFactory`. The original name `SessionDispatcherFactory` seems related to `Sess

Re: [PR] Add support for GaussDB database type [flink-connector-jdbc]

2025-04-21 Thread via GitHub
liubao68 commented on code in PR #162: URL: https://github.com/apache/flink-connector-jdbc/pull/162#discussion_r2052340905 ## flink-connector-jdbc-gaussdb/pom.xml: ## @@ -0,0 +1,135 @@ + Review Comment: Add license header -- This is an automated message from the Apache

[jira] [Updated] (FLINK-37706) [Flink CDC] Oceanbase OceanBaseMetadataApplierTest failed due to Caused by: java.sql.SQLException: Server out of disk space

2025-04-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37706: --- Labels: pull-request-available (was: ) > [Flink CDC] Oceanbase OceanBaseMetadataApplierTest

[jira] [Commented] (FLINK-37706) [Flink CDC] Oceanbase OceanBaseMetadataApplierTest failed due to Caused by: java.sql.SQLException: Server out of disk space

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17946108#comment-17946108 ] Leonard Xu commented on FLINK-37706: We used latest OB docker version in test, but r

Re: [PR] [FLINK-37697][runtime] Fix application mode cluster shut down before … [flink]

2025-04-21 Thread via GitHub
flinkbot commented on PR #26489: URL: https://github.com/apache/flink/pull/26489#issuecomment-2818252037 ## CI report: * 645739ae4eb8d9fad870136de6a4f66dfb48607f UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Assigned] (FLINK-37706) [Flink CDC] Oceanbase OceanBaseMetadataApplierTest failed due to Caused by: java.sql.SQLException: Server out of disk space

2025-04-21 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-37706: -- Assignee: Leonard Xu > [Flink CDC] Oceanbase OceanBaseMetadataApplierTest failed due to Cause

Re: [PR] [FLINK-29549]- Flink Glue Catalog integration [flink-connector-aws]

2025-04-21 Thread via GitHub
FranMorilloAWS commented on code in PR #191: URL: https://github.com/apache/flink-connector-aws/pull/191#discussion_r2052295570 ## flink-catalog-aws-glue/src/main/java/org/apache/flink/table/catalog/glue/operations/GlueTableOperations.java: ## @@ -0,0 +1,255 @@ +/* + * Licensed

[jira] [Updated] (FLINK-37697) ExecutionGraphInfo may not be archived when cluster shutdown in application mode

2025-04-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37697: --- Labels: pull-request-available (was: ) > ExecutionGraphInfo may not be archived when cluste

[PR] [FLINK-37697][runtime] Fix application mode cluster shut down before … [flink]

2025-04-21 Thread via GitHub
eemario opened a new pull request, #26489: URL: https://github.com/apache/flink/pull/26489 …all job termination futures complete ## What is the purpose of the change *This pull request makes the cluster shutdown after all job termination futures complete in applicat

Re: [PR] [FLINK-37705][rest] JobPlanInfo properties spec adjustments [flink]

2025-04-21 Thread via GitHub
Efrat19 commented on PR #26488: URL: https://github.com/apache/flink/pull/26488#issuecomment-2818239895 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xuyangzhong commented on code in PR #26379: URL: https://github.com/apache/flink/pull/26379#discussion_r2052281571 ## flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/batch/sql/BatchSinkReuseTest.xml: ## @@ -0,0 +1,243 @@ + + + + +

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xuyangzhong commented on PR #26379: URL: https://github.com/apache/flink/pull/26379#issuecomment-2818218602 @xiangyuf Hi, thanks for updating. I just some other minor comments. BTW, @davidradl do you have more thoughts about this pr? -- This is an automated message from the Apache Git Ser

[jira] [Updated] (FLINK-37708) Print digest about SinkAbilitySpec in plan just like SourceAbilitySpec

2025-04-21 Thread Xuyang Zhong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuyang Zhong updated FLINK-37708: - Affects Version/s: 1.20.1 1.19.2 2.0.0 > Print dig

[jira] [Updated] (FLINK-37708) Print digest about SinkAbilitySpec in plan just like SourceAbilitySpec

2025-04-21 Thread Xuyang Zhong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuyang Zhong updated FLINK-37708: - Component/s: Table SQL / Planner > Print digest about SinkAbilitySpec in plan just like SourceAb

[jira] [Updated] (FLINK-37708) Print digest about SinkAbilitySpec in plan just like SourceAbilitySpec

2025-04-21 Thread Xuyang Zhong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuyang Zhong updated FLINK-37708: - Description: [https://github.com/apache/flink/pull/26379#discussion_r2048105086]   Maybe we shou

[jira] [Created] (FLINK-37708) Print digest about SinkAbilitySpec in plan just like SourceAbilitySpec

2025-04-21 Thread Xuyang Zhong (Jira)
Xuyang Zhong created FLINK-37708: Summary: Print digest about SinkAbilitySpec in plan just like SourceAbilitySpec Key: FLINK-37708 URL: https://issues.apache.org/jira/browse/FLINK-37708 Project: Flink

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xuyangzhong commented on code in PR #26379: URL: https://github.com/apache/flink/pull/26379#discussion_r2052275668 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/reuse/SinkReuser.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Soft

Re: [PR] [FLINK-37543][table-planner] Support sink reuse in batch mode [flink]

2025-04-21 Thread via GitHub
xuyangzhong commented on code in PR #26379: URL: https://github.com/apache/flink/pull/26379#discussion_r2052274380 ## flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/reuse/SinkReuser.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Soft

Re: [PR] [FLINK-37132][transform] validate meta schema in Multi Transform [flink-cdc]

2025-04-21 Thread via GitHub
lvyanquan commented on PR #3865: URL: https://github.com/apache/flink-cdc/pull/3865#issuecomment-2818203231 Thanks for your fix. But there is now a user scenario where users want to add fields of `__namespace_name__ || '.' || __schema_name__ || '.' || __table_name__ identifier_name` and

Re: [PR] Handle dwh delete op [flink-connector-jdbc]

2025-04-21 Thread via GitHub
hieu2102 closed pull request #163: Handle dwh delete op URL: https://github.com/apache/flink-connector-jdbc/pull/163 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] Handle dwh delete op [flink-connector-jdbc]

2025-04-21 Thread via GitHub
boring-cyborg[bot] commented on PR #163: URL: https://github.com/apache/flink-connector-jdbc/pull/163#issuecomment-2818188413 Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html) -- This is

[PR] Handle dwh delete op [flink-connector-jdbc]

2025-04-21 Thread via GitHub
hieu2102 opened a new pull request, #163: URL: https://github.com/apache/flink-connector-jdbc/pull/163 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] [FLINK-37542][table-planner] Support sink reuse in streaming mode [flink]

2025-04-21 Thread via GitHub
xiangyuf commented on PR #26380: URL: https://github.com/apache/flink/pull/26380#issuecomment-2818179237 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [FLINK-37132][transform] validate meta schema in Multi Transform [flink-cdc]

2025-04-21 Thread via GitHub
MOBIN-F commented on PR #3865: URL: https://github.com/apache/flink-cdc/pull/3865#issuecomment-2818133148 @yuxiqian @lvyanquan PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] [FLINK-29549]- Flink Glue Catalog integration [flink-connector-aws]

2025-04-21 Thread via GitHub
FranMorilloAWS commented on code in PR #191: URL: https://github.com/apache/flink-connector-aws/pull/191#discussion_r2052228672 ## flink-catalog-aws-glue/src/main/java/org/apache/flink/table/catalog/glue/GlueCatalog.java: ## @@ -0,0 +1,925 @@ +/* + * Licensed to the Apache Soft

Re: [PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

2025-04-21 Thread via GitHub
yuxiqian commented on PR #3887: URL: https://github.com/apache/flink-cdc/pull/3887#issuecomment-2818105857 Minor suggestion: shall we pick `MySqlToPaimonE2eITCase` and `FlinkParallelizedPipelineITCase` to be executed with different source / sink parallelism? I believe they could cover most

Re: [PR] [FLINK-35713][common] Support set parallelism in source and sink [flink-cdc]

2025-04-21 Thread via GitHub
leonardBang commented on code in PR #3887: URL: https://github.com/apache/flink-cdc/pull/3887#discussion_r2052201271 ## flink-cdc-cli/src/main/java/org/apache/flink/cdc/cli/parser/YamlPipelineDefinitionParser.java: ## @@ -188,7 +189,14 @@ private SourceDef toSourceDef(JsonNode s

  1   2   >