Re: [PR] [FLINK-36694][Filesystems] Update Hadoop for filesystems to 3.4.1 [flink]

2025-01-22 Thread via GitHub
davidradl commented on PR #26046: URL: https://github.com/apache/flink/pull/26046#issuecomment-2606698870 I am wondering what the story is for master as this is against 120? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[PR] [FLINK-37138][sql-gateway] Fix failed SqlYARNApplicationITCase in hadoop3 profile [flink]

2025-01-22 Thread via GitHub
fsk119 opened a new pull request, #26050: URL: https://github.com/apache/flink/pull/26050 ## What is the purpose of the change *YARN deployment doesn't the directories as flink-dist does. Actually, it uses the following structures* ``` . └── ohmeatball ├──

Re: [PR] [FLINK-35139][Connectors/MongoDB] Drop support for Flink 1.18 and update development version to 2.0-SNAPSHOT [flink-connector-mongodb]

2025-01-22 Thread via GitHub
Jiabao-Sun commented on PR #48: URL: https://github.com/apache/flink-connector-mongodb/pull/48#issuecomment-2606702143 Hi @leonardBang, could you help review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] [hotfix][docs] Fix dead links in Doris [flink-cdc]

2025-01-22 Thread via GitHub
ruanhang1993 merged PR #3878: URL: https://github.com/apache/flink-cdc/pull/3878 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flin

[jira] [Updated] (FLINK-37138) testDeployScriptViaSqlClient failed on AZP

2025-01-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37138: --- Labels: pull-request-available (was: ) > testDeployScriptViaSqlClient failed on AZP > -

[jira] (FLINK-37138) testDeployScriptViaSqlClient failed on AZP

2025-01-22 Thread Shengkai Fang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37138 ] Shengkai Fang deleted comment on FLINK-37138: --- was (Author: fsk119): [~Weijie Guo] Please use the origin issue to track the same problem > testDeployScriptViaSqlClient failed on AZP >

[PR] [FLINK-37138][sql-gateway] Fix failed SqlYARNApplicationITCase in hadoop3 profile [flink]

2025-01-22 Thread via GitHub
fsk119 opened a new pull request, #26049: URL: https://github.com/apache/flink/pull/26049 ## What is the purpose of the change *YARN deployment doesn't the directories as flink-dist does. Actually, it uses the following structures* ``` . └── ohmeatball ├──

[jira] [Created] (FLINK-37198) AvroToRowDataConverters factory omits legacyTimestampMapping option for nested Row.

2025-01-22 Thread Juliusz Nadberezny (Jira)
Juliusz Nadberezny created FLINK-37198: -- Summary: AvroToRowDataConverters factory omits legacyTimestampMapping option for nested Row. Key: FLINK-37198 URL: https://issues.apache.org/jira/browse/FLINK-37198

Re: [PR] mention scylladb in docs [flink-connector-cassandra]

2025-01-22 Thread via GitHub
guy9 commented on PR #32: URL: https://github.com/apache/flink-connector-cassandra/pull/32#issuecomment-2606881550 Also interested in this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[jira] [Updated] (FLINK-37201) FlinkCDC Pipeline transform arithmetic functions support parameters of null and more numerical types.

2025-01-22 Thread Wenkai Qi (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenkai Qi updated FLINK-37201: -- Description: FlinkCDC Pipeline transform arithmetic functions can not support parameters of null and

Re: [PR] [BP-2.0][FLINK-36838][state/forst] Fix the deadlock when quit forst state backend [flink]

2025-01-22 Thread via GitHub
flinkbot commented on PR #26054: URL: https://github.com/apache/flink/pull/26054#issuecomment-2607441606 ## CI report: * 83a855a5b573f09ea590ca499b54b6410bb45478 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
ferenc-csaky commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607471427 @He-Pin That's a good point regarding cache! Probably the CI machines has less than 10 cores. I can give it a try, I just have to do a local Pekko build myself to be able to build Fl

Re: [PR] [FLINK-37175][table] Support JSON built-in function for JSON_OBJECT [flink]

2025-01-22 Thread via GitHub
gustavodemorais commented on code in PR #26022: URL: https://github.com/apache/flink/pull/26022#discussion_r1925151673 ## flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/Expressions.java: ## @@ -861,9 +862,13 @@ public static ApiExpression withoutColumn

Re: [PR] [FLINK-37021][state/forst] Fix incorrect paths when reusing files for checkpoints. [flink]

2025-01-22 Thread via GitHub
AlexYinHan commented on code in PR #26040: URL: https://github.com/apache/flink/pull/26040#discussion_r1925152740 ## flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/datatransfer/CopyDataTransferStrategy.java: ## @@ -92,97 +89,85 @@ public

Re: [PR] [FLINK-37021][state/forst] Fix incorrect paths when reusing files for checkpoints. [flink]

2025-01-22 Thread via GitHub
AlexYinHan commented on code in PR #26040: URL: https://github.com/apache/flink/pull/26040#discussion_r1925152164 ## flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/datatransfer/CopyDataTransferStrategy.java: ## @@ -92,97 +89,85 @@ public

Re: [PR] [FLINK-37021][state/forst] Fix incorrect paths when reusing files for checkpoints. [flink]

2025-01-22 Thread via GitHub
AlexYinHan commented on code in PR #26040: URL: https://github.com/apache/flink/pull/26040#discussion_r1925154975 ## flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/datatransfer/DataTransferStrategyBuilder.java: ## @@ -45,20 +45,25 @@ pub

[jira] [Updated] (FLINK-37199) Update document in quickly start to guide user to add checkpoint configuration

2025-01-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37199: --- Labels: pull-request-available (was: ) > Update document in quickly start to guide user to

[PR] [FLINK-37199][doc] Update configuration in quickstart. [flink-cdc]

2025-01-22 Thread via GitHub
lvyanquan opened a new pull request, #3880: URL: https://github.com/apache/flink-cdc/pull/3880 Update configuration file for Flink 1.19+. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] [FLINK-36564][ci] Running CI in random timezone to expose more time related bugs. [flink-cdc]

2025-01-22 Thread via GitHub
lvyanquan commented on PR #3650: URL: https://github.com/apache/flink-cdc/pull/3650#issuecomment-2606981241 Hi @leonardBang, could you help to reopen it as this will help us find more potential bugs. -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] [FLINK-37175][table] Support JSON built-in function for JSON_OBJECT [flink]

2025-01-22 Thread via GitHub
gustavodemorais commented on PR #26022: URL: https://github.com/apache/flink/pull/26022#issuecomment-2606982236 Made the changes as we discussed @twalthr. One additional thing I've changed is that we not only parse the json, but convert the json back to string before storing it. I think tha

Re: [PR] [FLINK-36838][state/forst] Fix the deadlock when quit forst state backend [flink]

2025-01-22 Thread via GitHub
flinkbot commented on PR #26053: URL: https://github.com/apache/flink/pull/26053#issuecomment-2607233199 ## CI report: * baf5b7d55de9f79a93d7356d9c4b3d4574886845 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [BP-2.0][FLINK-37183][clients] Fix symbolic links not being followed in "usrlib" [flink]

2025-01-22 Thread via GitHub
flinkbot commented on PR #26052: URL: https://github.com/apache/flink/pull/26052#issuecomment-2607231281 ## CI report: * 900a5e229422238ee0c3a8e6652b81574880d27a UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Assigned] (FLINK-37191) MySQL 'Compile and test' stage timeout after 90 minutes

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-37191: -- Assignee: yux > MySQL 'Compile and test' stage timeout after 90 minutes > ---

[PR] [FLINK-36564][ci] Running CI in random timezone to expose more time related bugs. [flink-cdc]

2025-01-22 Thread via GitHub
lvyanquan opened a new pull request, #3650: URL: https://github.com/apache/flink-cdc/pull/3650 Refer to [this comment](https://github.com/apache/flink-cdc/pull/3648#pullrequestreview-2374769012) Copy from Paimon repo. How to verify: ![image](https://github.com/user-attachments/

Re: [PR] [FLINK-36694][Filesystems] Update Hadoop for filesystems to 3.4.1 [flink]

2025-01-22 Thread via GitHub
bbrandt commented on PR #26046: URL: https://github.com/apache/flink/pull/26046#issuecomment-2607635603 Thanks @davidradl. Mostly using this as a temporary working area to see what the blockers are with the change in 1.20. If it high enough priority enough to move forward with, I can cher

[jira] [Created] (FLINK-37203) Support AltertableCommentEvent and AlterColumnCommentEvent.

2025-01-22 Thread Yanquan Lv (Jira)
Yanquan Lv created FLINK-37203: -- Summary: Support AltertableCommentEvent and AlterColumnCommentEvent. Key: FLINK-37203 URL: https://issues.apache.org/jira/browse/FLINK-37203 Project: Flink Issu

Re: [PR] [FLINK-37021][state/forst] Fix incorrect paths when reusing files for checkpoints. [flink]

2025-01-22 Thread via GitHub
davidradl commented on code in PR #26040: URL: https://github.com/apache/flink/pull/26040#discussion_r1924951833 ## flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/datatransfer/CopyDataTransferStrategy.java: ## @@ -92,97 +89,85 @@ public

Re: [PR] [FLINK-37181][build] Fix MacOS sha256sum tool compile error [flink]

2025-01-22 Thread via GitHub
gaborgsomogyi commented on PR #26034: URL: https://github.com/apache/flink/pull/26034#issuecomment-2606656369 Some unrelated tests are flaky so restarting. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [FLINK-37181][build] Fix MacOS sha256sum tool compile error [flink]

2025-01-22 Thread via GitHub
gaborgsomogyi commented on PR #26034: URL: https://github.com/apache/flink/pull/26034#issuecomment-2606656815 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] [FLINK-37021][state/forst] Fix incorrect paths when reusing files for checkpoints. [flink]

2025-01-22 Thread via GitHub
davidradl commented on code in PR #26040: URL: https://github.com/apache/flink/pull/26040#discussion_r1924954408 ## flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/datatransfer/CopyDataTransferStrategy.java: ## @@ -92,97 +89,85 @@ public

Re: [PR] [FLINK-37120][pipeline-connector/mysql] Add ending split chunk first to avoid TaskManager oom [flink-cdc]

2025-01-22 Thread via GitHub
lvyanquan commented on PR #3856: URL: https://github.com/apache/flink-cdc/pull/3856#issuecomment-2607006968 LGTM. I'm worried if this parameter is a bit complicated, what about naming it `scan.incremental.assign-max-chunk-first.enabled`? WDYT @beryllw @leonardBang? -- This is an aut

[jira] [Created] (FLINK-37201) FlinkCDC Pipeline transform arithmetic functions support parameters of null and more numerical types.

2025-01-22 Thread Wenkai Qi (Jira)
Wenkai Qi created FLINK-37201: - Summary: FlinkCDC Pipeline transform arithmetic functions support parameters of null and more numerical types. Key: FLINK-37201 URL: https://issues.apache.org/jira/browse/FLINK-37201

Re: [PR] [FLINK-36698][pipeline-connector][elasticsearch] Elasticsearch Pipeline Connector support index sharding [flink-cdc]

2025-01-22 Thread via GitHub
lvyanquan commented on code in PR #3723: URL: https://github.com/apache/flink-cdc/pull/3723#discussion_r1925186674 ## docs/content/docs/connectors/pipeline-connectors/elasticsearch.md: ## @@ -169,7 +169,7 @@ Pipeline Connector Options optional (none) Long -

Re: [PR] [FLINK-35600] Add timestamp for low and high watermark [flink-cdc]

2025-01-22 Thread via GitHub
yuxiqian commented on PR #3415: URL: https://github.com/apache/flink-cdc/pull/3415#issuecomment-2607045316 After some bisect searching I believe changes in this PR will caused PolarDBX related tests to fail frequently (FLINK-37191). -- This is an automated message from the Apache Git Serv

Re: [PR] [FLINK-36973]udf dateformat support LocalZonedTimestampData datatype [flink-cdc]

2025-01-22 Thread via GitHub
hiliuxg commented on code in PR #3863: URL: https://github.com/apache/flink-cdc/pull/3863#discussion_r1925357513 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/functions/SystemFunctionUtils.java: ## @@ -105,6 +105,13 @@ public static String dateFormat(TimestampD

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
He-Pin commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607651794 ouch, `but Flink does not support 2.13 at all` so seems we can't drop 2.12 support very soon! -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
ferenc-csaky commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607710502 > ouch, `but Flink does not support 2.13 at all` so seems we can't drop 2.12 support very soon! We are actually trying to remove Scala from the project: https://issues.apache.

[jira] [Created] (FLINK-37199) Update document in quickly start to guide user to add checkpoint configuration

2025-01-22 Thread Yanquan Lv (Jira)
Yanquan Lv created FLINK-37199: -- Summary: Update document in quickly start to guide user to add checkpoint configuration Key: FLINK-37199 URL: https://issues.apache.org/jira/browse/FLINK-37199 Project: F

Re: [PR] [FLINK-37138][sql-gateway] Fix failed SqlYARNApplicationITCase in hadoop3 profile [flink]

2025-01-22 Thread via GitHub
fsk119 commented on PR #26049: URL: https://github.com/apache/flink/pull/26049#issuecomment-2606948334 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] [FLINK-37138][sql-gateway] Fix failed SqlYARNApplicationITCase in hadoop3 profile [flink]

2025-01-22 Thread via GitHub
fsk119 commented on PR #26050: URL: https://github.com/apache/flink/pull/26050#issuecomment-2606948717 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] [FLINK-20895][flink-table-planner-blink] support local aggregate push down in blink planner [flink]

2025-01-22 Thread via GitHub
sebastianliu closed pull request #14894: [FLINK-20895][flink-table-planner-blink] support local aggregate push down in blink planner URL: https://github.com/apache/flink/pull/14894 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Commented] (FLINK-37177) Update chinese documentation of json function

2025-01-22 Thread qinghuan wang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916092#comment-17916092 ] qinghuan wang commented on FLINK-37177: --- hi, can you assign this one to me? [~gust

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
ferenc-csaky commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607584839 > @ferenc-csaky Cool But the pekko 1.2.x snapshot should be binary compatible with 1.1.x > > And there are nightly on https://repository.apache.org/content/groups/snapshots/or

Re: [PR] [hotfix] Fix the issue related to mounting the Logback configuration [flink]

2025-01-22 Thread via GitHub
Jam804 commented on PR #26039: URL: https://github.com/apache/flink/pull/26039#issuecomment-2607614914 I think there's nothing wrong with my code now. Thank you for your guidance. @davidradl -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[PR] [FLINK-36868][state/forst] Fix the deadlock when quit forst state backend [flink]

2025-01-22 Thread via GitHub
Zakelly opened a new pull request, #26053: URL: https://github.com/apache/flink/pull/26053 ## What is the purpose of the change In #25732 we join the async background threads of ForSt to avoid deadlock when JVM quit. That is an incomplete solution which is also problematic. The backg

Re: [PR] [FLINK-37138][sql-gateway] Fix failed SqlYARNApplicationITCase in hadoop3 profile [flink]

2025-01-22 Thread via GitHub
flinkbot commented on PR #26049: URL: https://github.com/apache/flink/pull/26049#issuecomment-2607036091 ## CI report: * c64f52cdc11078f135a632742918afcbfc0a857a UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-37138][sql-gateway] Fix failed SqlYARNApplicationITCase in hadoop3 profile [flink]

2025-01-22 Thread via GitHub
flinkbot commented on PR #26050: URL: https://github.com/apache/flink/pull/26050#issuecomment-2607037853 ## CI report: * e616aef00b98f25ca1a60c0aca730e0b257fce09 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-36698][pipeline-connector][elasticsearch] Elasticsearch Pipeline Connector support index sharding [flink-cdc]

2025-01-22 Thread via GitHub
lvyanquan commented on code in PR #3723: URL: https://github.com/apache/flink-cdc/pull/3723#discussion_r1925186674 ## docs/content/docs/connectors/pipeline-connectors/elasticsearch.md: ## @@ -169,7 +169,7 @@ Pipeline Connector Options optional (none) Long -

Re: [PR] [FLINK-37005][table] Make StreamExecDeduplicate ouput insert only where possible [flink]

2025-01-22 Thread via GitHub
flinkbot commented on PR #26051: URL: https://github.com/apache/flink/pull/26051#issuecomment-2607039595 ## CI report: * 63f9ed19290ca306170c67664f67750ce8d6e8e3 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Created] (FLINK-37200) Add e2e test for ElasticSearch Sink.

2025-01-22 Thread Yanquan Lv (Jira)
Yanquan Lv created FLINK-37200: -- Summary: Add e2e test for ElasticSearch Sink. Key: FLINK-37200 URL: https://issues.apache.org/jira/browse/FLINK-37200 Project: Flink Issue Type: Improvement

[PR] [BP-2.0][FLINK-36838][state/forst] Fix the deadlock when quit forst state backend [flink]

2025-01-22 Thread via GitHub
Zakelly opened a new pull request, #26054: URL: https://github.com/apache/flink/pull/26054 This is a backport PR to 2.0 for #26053 ## What is the purpose of the change In #25732 we join the async background threads of ForSt to avoid deadlock when JVM quit. That is an incomplet

[jira] [Updated] (FLINK-37201) FlinkCDC Pipeline transform arithmetic functions support parameters of null and more numerical types.

2025-01-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37201: --- Labels: pull-request-available (was: ) > FlinkCDC Pipeline transform arithmetic functions s

[jira] [Updated] (FLINK-37177) Update chinese documentation of json function

2025-01-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37177: --- Labels: chinese-translation pull-request-available (was: chinese-translation) > Update chi

[PR] [FLINK-37177][table] Update chinese documentation of json function [flink]

2025-01-22 Thread via GitHub
wangqinghuan opened a new pull request, #26055: URL: https://github.com/apache/flink/pull/26055 ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring th

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
He-Pin commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607565950 @ferenc-csaky Cool But the pekko 1.2.x snapshot should be binary compatible with 1.1.x And there are nightly on https://repository.apache.org/content/groups/snapshots/org/apache/p

Re: [PR] Update cdc version to 3.4-SNAPSHOT and add release-3.3 docs [flink-cdc]

2025-01-22 Thread via GitHub
leonardBang merged PR #3870: URL: https://github.com/apache/flink-cdc/pull/3870 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [docs] Fix maxcompute connector typo in examples section [flink-cdc]

2025-01-22 Thread via GitHub
leonardBang merged PR #3875: URL: https://github.com/apache/flink-cdc/pull/3875 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [docs] Fix maxcompute connector typo in examples section [flink-cdc]

2025-01-22 Thread via GitHub
leonardBang commented on PR #3875: URL: https://github.com/apache/flink-cdc/pull/3875#issuecomment-2606733305 CI passed, merging... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] [FLINK-37005][table] Make StreamExecDeduplicate ouput insert only where possible [flink]

2025-01-22 Thread via GitHub
pnowojski commented on PR #26051: URL: https://github.com/apache/flink/pull/26051#issuecomment-2606730242 @lincoln-lil @xuyangzhong can you take a look at this? You were recently involved in a couple of changes around here: FLINK-36837 and FLINK-34702. I could try to implement the asy

[jira] [Created] (FLINK-37202) Introduce TDengine connector

2025-01-22 Thread kevinmen (Jira)
kevinmen created FLINK-37202: Summary: Introduce TDengine connector Key: FLINK-37202 URL: https://issues.apache.org/jira/browse/FLINK-37202 Project: Flink Issue Type: New Feature Repo

[jira] [Commented] (FLINK-37202) Introduce TDengine connector

2025-01-22 Thread David Radley (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916044#comment-17916044 ] David Radley commented on FLINK-37202: -- Hi there, I think a new connector is an imp

[PR] [hotfix][docs] Fix dead links in Doris [flink-cdc]

2025-01-22 Thread via GitHub
ruanhang1993 opened a new pull request, #3878: URL: https://github.com/apache/flink-cdc/pull/3878 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [PR] [FLINK-36945] MySQL CDC supports parsing RENAME TABLE Statement with multiple tables [flink-cdc]

2025-01-22 Thread via GitHub
yoheimuta commented on PR #3876: URL: https://github.com/apache/flink-cdc/pull/3876#issuecomment-2606833211 @Mielientiev It's similar but not related. The fixed PR for #3668 does not address the issue presented here. And, I’m unsure if this fix resolves potential issues with gh-ost or pt-os

Re: [PR] [FLINK-36945] MySQL CDC supports parsing RENAME TABLE Statement with multiple tables [flink-cdc]

2025-01-22 Thread via GitHub
mielientiev commented on PR #3876: URL: https://github.com/apache/flink-cdc/pull/3876#issuecomment-2606818887 Quick question isn't this related to https://github.com/apache/flink-cdc/pull/3668? -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] [FLINK-37194][tests] Fix NPE issue in WatermarkITCase [flink]

2025-01-22 Thread via GitHub
flinkbot commented on PR #26048: URL: https://github.com/apache/flink/pull/26048#issuecomment-2606839429 ## CI report: * c227934d08287a0311e85f563c2026da5dff452e UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Updated] (FLINK-37005) Make StreamExecDeduplicate ouput insert only where possible

2025-01-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37005: --- Labels: pull-request-available (was: ) > Make StreamExecDeduplicate ouput insert only where

[PR] [FLINK-37005][table] Make StreamExecDeduplicate ouput insert only where possible [flink]

2025-01-22 Thread via GitHub
pnowojski opened a new pull request, #26051: URL: https://github.com/apache/flink/pull/26051 ## What is the purpose of the change This PR: - implements the `RowTimeDeduplicateKeepFirstRowFunction` using watermarks to avoid retracting previous results - changes planner to take in

Re: [PR] [FLINK-37021][state/forst] Fix incorrect paths when reusing files for checkpoints. [flink]

2025-01-22 Thread via GitHub
Zakelly commented on code in PR #26040: URL: https://github.com/apache/flink/pull/26040#discussion_r1925014517 ## flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/datatransfer/DataTransferStrategyBuilder.java: ## @@ -45,20 +45,25 @@ public

Re: [PR] [FLINK-37175][table] Support JSON built-in function for JSON_OBJECT [flink]

2025-01-22 Thread via GitHub
gustavodemorais commented on PR #26022: URL: https://github.com/apache/flink/pull/26022#issuecomment-2607001419 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] [FLINK-35578][state/rocksdb] Bump frocksdbjni to 8.10.0-ververica-1.0 [flink]

2025-01-22 Thread via GitHub
fredia commented on PR #26033: URL: https://github.com/apache/flink/pull/26033#issuecomment-2607013048 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
He-Pin commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607379254 @ferenc-csaky I think the cache depends on how many cores you have, would you like to test the pekko 1.2.x nightly too, I would like too, but don't know how. refs: https://github.com/ap

Re: [PR] [FLINK-37175][table] Support JSON built-in function for JSON_OBJECT [flink]

2025-01-22 Thread via GitHub
gustavodemorais closed pull request #26022: [FLINK-37175][table] Support JSON built-in function for JSON_OBJECT URL: https://github.com/apache/flink/pull/26022 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
ferenc-csaky commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607368545 @afedulov So on my M4 Pro chip MacBook the test starts to run with 11MB consistentlt, and consistently fail with anything under that, since TMs cannot even start because there are no

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
He-Pin commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607796620 I knew , after alibaba acquired, then more Java, thanks for that information. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
ferenc-csaky commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607825560 > @He-Pin now that it is confirmed that the "unpooled" config solves the issue for us, how realistic do you think it is to get a Pekko release with the allocator config support inclu

Re: [PR] [FLINK-36694][Filesystems] Update Hadoop for filesystems to 3.4.1 [flink]

2025-01-22 Thread via GitHub
bbrandt closed pull request #26046: [FLINK-36694][Filesystems] Update Hadoop for filesystems to 3.4.1 URL: https://github.com/apache/flink/pull/26046 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] [BP-2.0][FLINK-37183][clients] Fix symbolic links not being followed in "usrlib" [flink]

2025-01-22 Thread via GitHub
ferenc-csaky closed pull request #26052: [BP-2.0][FLINK-37183][clients] Fix symbolic links not being followed in "usrlib" URL: https://github.com/apache/flink/pull/26052 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] [BP-2.0][FLINK-37183][clients] Fix symbolic links not being followed in "usrlib" [flink]

2025-01-22 Thread via GitHub
ferenc-csaky commented on PR #26052: URL: https://github.com/apache/flink/pull/26052#issuecomment-2607997344 The `flink-clients` test module ran correctly, merging this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Comment Edited] (FLINK-37183) Usrlib symlinks are not followed

2025-01-22 Thread Ferenc Csaky (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17915937#comment-17915937 ] Ferenc Csaky edited comment on FLINK-37183 at 1/22/25 6:54 PM: ---

[jira] [Comment Edited] (FLINK-33926) Can't start a job with a jar in the system classpath in native k8s mode

2025-01-22 Thread Trystan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916171#comment-17916171 ] Trystan edited comment on FLINK-33926 at 1/22/25 7:32 PM: -- I th

[jira] [Commented] (FLINK-33926) Can't start a job with a jar in the system classpath in native k8s mode

2025-01-22 Thread Trystan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916171#comment-17916171 ] Trystan commented on FLINK-33926: - I think it makes sense to me. Not sure how something

Re: [PR] [BP-2.0][FLINK-37099][docs] Document memory/performance considerations and config possibilities for Netty4 [flink]

2025-01-22 Thread via GitHub
flinkbot commented on PR #26056: URL: https://github.com/apache/flink/pull/26056#issuecomment-2608371874 ## CI report: * ea7a81374111aed2e7869ef3e8860e7fe41146e1 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-37177][Doc] Update chinese documentation of json function [flink]

2025-01-22 Thread via GitHub
flinkbot commented on PR #26055: URL: https://github.com/apache/flink/pull/26055#issuecomment-2607876888 ## CI report: * 3f2569c82e8b1ba329f9a25dd86fb883ae5f11b7 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-36698][pipeline-connector][elasticsearch] Elasticsearch Pipeline Connector support index sharding [flink-cdc]

2025-01-22 Thread via GitHub
beryllw commented on code in PR #3723: URL: https://github.com/apache/flink-cdc/pull/3723#discussion_r1926193705 ## docs/content/docs/connectors/pipeline-connectors/elasticsearch.md: ## @@ -169,7 +169,7 @@ Pipeline Connector Options optional (none) Long -

Re: [PR] [FLINK-36698][pipeline-connector][elasticsearch] Elasticsearch Pipeline Connector support index sharding [flink-cdc]

2025-01-22 Thread via GitHub
beryllw commented on code in PR #3723: URL: https://github.com/apache/flink-cdc/pull/3723#discussion_r1926193705 ## docs/content/docs/connectors/pipeline-connectors/elasticsearch.md: ## @@ -169,7 +169,7 @@ Pipeline Connector Options optional (none) Long -

[jira] [Commented] (FLINK-32212) Job restarting indefinitely after an IllegalStateException from BlobLibraryCacheManager

2025-01-22 Thread Vineeth Naroju (Jira)
[ https://issues.apache.org/jira/browse/FLINK-32212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916238#comment-17916238 ] Vineeth Naroju commented on FLINK-32212: We encountered same error just now in F

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
He-Pin commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607805457 Would you like to comment on the pr against 1.1.x too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
afedulov commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607804741 @He-Pin now that it is confirmed that the "unpooled" config solves the issue for us, how realistic do you think it is to get a Pekko release with the allocator config support included so

[jira] [Updated] (FLINK-37148) Add documentation for datastream v2 api

2025-01-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37148: --- Labels: pull-request-available (was: ) > Add documentation for datastream v2 api >

[jira] [Assigned] (FLINK-37201) FlinkCDC Pipeline transform arithmetic functions support parameters of null and more numerical types.

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-37201: -- Assignee: Wenkai Qi > FlinkCDC Pipeline transform arithmetic functions support parameters of

[jira] [Assigned] (FLINK-37200) Add e2e test for ElasticSearch Sink.

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-37200: -- Assignee: Yanquan Lv > Add e2e test for ElasticSearch Sink. > ---

[jira] [Commented] (FLINK-35671) Support Iceberg CDC Pipeline Sink

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916246#comment-17916246 ] Leonard Xu commented on FLINK-35671: Hey [~m.orazow] , do you have time to work for

Re: [PR] Draft: [FLINK-36061] Support Iceberg CDC Pipeline SinkV2 [flink-cdc]

2025-01-22 Thread via GitHub
melin commented on PR #3877: URL: https://github.com/apache/flink-cdc/pull/3877#issuecomment-2608746353 Add compaction optimization? For example, commit 100 times and execute a compaction. Similar to this demos approach: https://github.com/aws-samples/iceberg-streaming-examples/blob/

Re: [PR] [FLINK-36698][pipeline-connector][elasticsearch] Elasticsearch Pipeline Connector support index sharding [flink-cdc]

2025-01-22 Thread via GitHub
beryllw commented on code in PR #3723: URL: https://github.com/apache/flink-cdc/pull/3723#discussion_r1926291799 ## docs/content/docs/connectors/pipeline-connectors/elasticsearch.md: ## @@ -169,7 +169,7 @@ Pipeline Connector Options optional (none) Long -

[jira] [Resolved] (FLINK-36061) Support Apache iceberg for pipeline connector

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-36061. Resolution: Duplicate > Support Apache iceberg for pipeline connector > --

[jira] [Commented] (FLINK-36061) Support Apache iceberg for pipeline connector

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916248#comment-17916248 ] Leonard Xu commented on FLINK-36061: [~rmoff]You're right, this is a duplicated issu

[jira] [Commented] (FLINK-36061) Support Apache iceberg for pipeline connector

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916250#comment-17916250 ] Leonard Xu commented on FLINK-36061: [~melin] +1 for your proposal, compaction optim

Re: [PR] [FLINK-37120][pipeline-connector/mysql] Add ending split chunk first to avoid TaskManager oom [flink-cdc]

2025-01-22 Thread via GitHub
beryllw commented on PR #3856: URL: https://github.com/apache/flink-cdc/pull/3856#issuecomment-2608748868 > LGTM. I'm worried if this parameter is a bit complicated, what about naming it `scan.incremental.assign-max-chunk-first.enabled` to avoid using `ending`? As `chunk` is a concept in sn

[jira] [Commented] (FLINK-36838) [state] Join background threads when ForSt state backend quit

2025-01-22 Thread Zakelly Lan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916258#comment-17916258 ] Zakelly Lan commented on FLINK-36838: - follow-up merge: * master: 7b2e9c410344126a26

[jira] [Updated] (FLINK-37161) Cross-team verification for "Adaptive skewed join optimization for batch jobs"

2025-01-22 Thread Junrui Lee (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junrui Lee updated FLINK-37161: --- Description: In Flink 2.0, we support the capability of adaptive skewed join optimization for batch

  1   2   >