Re: [PR] [WIP][FLINK-37148][docs] Add documents for DataStream V2 [flink]

2025-01-22 Thread via GitHub
reswqa commented on code in PR #26057: URL: https://github.com/apache/flink/pull/26057#discussion_r1926520983 ## docs/content/docs/dev/datastream-v2/building_blocks.md: ## @@ -0,0 +1,215 @@ +--- +title: Building Blocks +weight: 3 +type: docs +--- + + +# Building Blocks + +DataSt

Re: [PR] [FLINK-36698][pipeline-connector][elasticsearch] Elasticsearch Pipeline Connector support index sharding [flink-cdc]

2025-01-22 Thread via GitHub
beryllw commented on PR #3723: URL: https://github.com/apache/flink-cdc/pull/3723#issuecomment-2609091793 > LGTM. Thanks for your review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] [hotfix][docs] Change docs in master to 3.4-SNAPSHOT [flink-cdc]

2025-01-22 Thread via GitHub
ruanhang1993 merged PR #3886: URL: https://github.com/apache/flink-cdc/pull/3886 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flin

Re: [PR] [WIP][FLINK-37148][docs] Add documents for DataStream V2 [flink]

2025-01-22 Thread via GitHub
reswqa commented on code in PR #26057: URL: https://github.com/apache/flink/pull/26057#discussion_r1926517227 ## docs/content/docs/dev/datastream-v2/building_blocks.md: ## @@ -0,0 +1,215 @@ +--- +title: Building Blocks +weight: 3 +type: docs +--- + + +# Building Blocks + +DataSt

Re: [PR] [FLINK-36564][ci] Running CI in random timezone to expose more time related bugs. [flink-cdc]

2025-01-22 Thread via GitHub
lvyanquan commented on PR #3650: URL: https://github.com/apache/flink-cdc/pull/3650#issuecomment-2609084505 > @lvyanquan A rebase action is welcome Done. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[PR] [FLINK-37207][runtime] Don't clear distributed cache in StreamExecutionEnvironment.configure [flink]

2025-01-22 Thread via GitHub
dianfu opened a new pull request, #26062: URL: https://github.com/apache/flink/pull/26062 ## What is the purpose of the change *This pull request fixes the issue that the distributed cache was cleared in StreamExecutionEnvironment.configure. It has side effect that distributed caches

[jira] [Updated] (FLINK-37207) Distributed cache set in the DataStream API does not work if a job use both the Table API & DataStream API

2025-01-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37207: --- Labels: pull-request-available (was: ) > Distributed cache set in the DataStream API does n

[PR] [hotfix][docs] Change docs in master to 3.4-SNAPSHOT [flink-cdc]

2025-01-22 Thread via GitHub
ruanhang1993 opened a new pull request, #3886: URL: https://github.com/apache/flink-cdc/pull/3886 This PR changes docs in master to 3.4-SNAPSHOT -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[jira] [Updated] (FLINK-37207) Distributed cache set in the DataStream API does not work if a job use both the Table API & DataStream API

2025-01-22 Thread Dian Fu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dian Fu updated FLINK-37207: Attachment: image-2025-01-23-15-34-13-242.png > Distributed cache set in the DataStream API does not work

Re: [PR] [hotfix][docs] Fix dead links in Doris (#3878) [flink-cdc]

2025-01-22 Thread via GitHub
ruanhang1993 merged PR #3885: URL: https://github.com/apache/flink-cdc/pull/3885 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flin

[jira] [Updated] (FLINK-37207) Distributed cache set in the DataStream API does not work if a job use both the Table API & DataStream API

2025-01-22 Thread Dian Fu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dian Fu updated FLINK-37207: Description: It will clear the distributed cache in the method [StreamExecutionEnvironment.configure|http

[PR] [hotfix][docs] Fix dead links in Doris (#3878) [flink-cdc]

2025-01-22 Thread via GitHub
ruanhang1993 opened a new pull request, #3885: URL: https://github.com/apache/flink-cdc/pull/3885 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [PR] [FLINK-36698][pipeline-connector][elasticsearch] Elasticsearch Pipeline Connector support index sharding [flink-cdc]

2025-01-22 Thread via GitHub
lvyanquan commented on code in PR #3723: URL: https://github.com/apache/flink-cdc/pull/3723#discussion_r1926489573 ## flink-cdc-connect/flink-cdc-pipeline-connectors/flink-cdc-pipeline-connector-elasticsearch/src/main/java/org/apache/flink/cdc/connectors/elasticsearch/sink/Elasticsearch

[jira] [Created] (FLINK-37208) Properly notify a new key is selected for async state operators

2025-01-22 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-37208: --- Summary: Properly notify a new key is selected for async state operators Key: FLINK-37208 URL: https://issues.apache.org/jira/browse/FLINK-37208 Project: Flink

[jira] [Updated] (FLINK-37207) Distributed cache set in the DataStream API does not work if a job use both the Table API & DataStream API

2025-01-22 Thread Dian Fu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dian Fu updated FLINK-37207: Description: It will clear the distributed cache in the method [StreamExecutionEnvironment.configure|http

Re: [PR] [FLINK-36564][ci] Running CI in random timezone to expose more time related bugs. [flink-cdc]

2025-01-22 Thread via GitHub
leonardBang commented on PR #3650: URL: https://github.com/apache/flink-cdc/pull/3650#issuecomment-2609045244 @lvyanquan A rebase action is welcome -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Updated] (FLINK-37207) Distributed cache set in the DataStream API does not work if a job use both the Table API & DataStream API

2025-01-22 Thread Dian Fu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dian Fu updated FLINK-37207: Description: It will clear the distributed cache in the method[ StreamExecutionEnvironment.configure|http

Re: [PR] [FLINK-36698][pipeline-connector][elasticsearch] Elasticsearch Pipeline Connector support index sharding [flink-cdc]

2025-01-22 Thread via GitHub
lvyanquan commented on code in PR #3723: URL: https://github.com/apache/flink-cdc/pull/3723#discussion_r1926486699 ## docs/content/docs/connectors/pipeline-connectors/elasticsearch.md: ## @@ -169,7 +169,7 @@ Pipeline Connector Options optional (none) Long -

Re: [PR] [hotfix][build] Add release-3.3 docs build action [flink-cdc]

2025-01-22 Thread via GitHub
ruanhang1993 closed pull request #3879: [hotfix][build] Add release-3.3 docs build action URL: https://github.com/apache/flink-cdc/pull/3879 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[jira] [Created] (FLINK-37207) Distributed cache set in the DataStream API does not work if a job use both the Table API & DataStream API

2025-01-22 Thread Dian Fu (Jira)
Dian Fu created FLINK-37207: --- Summary: Distributed cache set in the DataStream API does not work if a job use both the Table API & DataStream API Key: FLINK-37207 URL: https://issues.apache.org/jira/browse/FLINK-37207

[PR] [hotfix] [docs] Fix code typo in task_failure_recovery.md [flink]

2025-01-22 Thread via GitHub
glorinli opened a new pull request, #26061: URL: https://github.com/apache/flink/pull/26061 ## What is the purpose of the change A minor typo fix in task_failure_recovery.md's java example code, the Duration is mispelled Durartion. ## Verifying this change Doc ch

[jira] [Created] (FLINK-37206) Initialization issue of batching timer service in async state operators

2025-01-22 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-37206: --- Summary: Initialization issue of batching timer service in async state operators Key: FLINK-37206 URL: https://issues.apache.org/jira/browse/FLINK-37206 Project: Flink

[jira] [Commented] (FLINK-36061) Support Apache iceberg for pipeline connector

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916295#comment-17916295 ] Leonard Xu commented on FLINK-36061: Splitting the connector development into two p

[jira] [Assigned] (FLINK-36061) Support Apache iceberg for pipeline connector

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-36061: -- Assignee: ZhengYu Chen > Support Apache iceberg for pipeline connector >

[jira] [Assigned] (FLINK-37204) Add missing StarRocks connector options for 1.2.10.

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-37204: -- Assignee: ChaomingZhang > Add missing StarRocks connector options for 1.2.10. > -

[PR] [tests] Improve CI test effectivness by isolating test matrix and unify timeout [flink-cdc]

2025-01-22 Thread via GitHub
yuxiqian opened a new pull request, #3884: URL: https://github.com/apache/flink-cdc/pull/3884 This is a preparatory PR (for FLINK-37191) mainly focusing on improving test case effectiveness, including: * Single-stage CI failure should not cause the whole matrix got cancelled * Adde

Re: [PR] [hotfix] Add profile to apply `--add-opens` for java17 CI [flink-benchmarks]

2025-01-22 Thread via GitHub
Zakelly commented on PR #103: URL: https://github.com/apache/flink-benchmarks/pull/103#issuecomment-2609010817 As we can see, the `--add-opens` is successfully added during CI testing. ![image](https://github.com/user-attachments/assets/21c684ad-e447-421c-b90d-01b6ae756d10) https://g

Re: [PR] [hotfix][cdc][docs] Build 3.3 docs and mark it as stable [flink-cdc]

2025-01-22 Thread via GitHub
ruanhang1993 merged PR #3882: URL: https://github.com/apache/flink-cdc/pull/3882 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flin

[PR] [FLINK-37205][python] Correct the state cache behavior during bump be… [flink]

2025-01-22 Thread via GitHub
dianfu opened a new pull request, #26060: URL: https://github.com/apache/flink/pull/26060 …am version ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-tr

[PR] [FLINK-37205][python] Correct the state cache behavior during bump be… [flink]

2025-01-22 Thread via GitHub
dianfu opened a new pull request, #26059: URL: https://github.com/apache/flink/pull/26059 …am version ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-tr

[jira] [Updated] (FLINK-37205) Correct the state cache behavior during bump beam version

2025-01-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37205: --- Labels: pull-request-available (was: ) > Correct the state cache behavior during bump beam

[PR] [FLINK-37205][python] Correct the state cache behavior during bump beam version [flink]

2025-01-22 Thread via GitHub
dianfu opened a new pull request, #26058: URL: https://github.com/apache/flink/pull/26058 ## What is the purpose of the change *This pull request fixes the state cache behavior during bump beam version. It has changed the state cache strategy from size based to bytes based since

[jira] [Updated] (FLINK-37205) Correct the state cache behavior during bump beam version

2025-01-22 Thread Dian Fu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dian Fu updated FLINK-37205: Description: It has changed the state cache strategy from size based to bytes based since Beam 2.42.0 (See

Re: [PR] [tests] Improve CI test effectivness by isolating test matrix and unify timeout [flink-cdc]

2025-01-22 Thread via GitHub
yuxiqian commented on PR #3884: URL: https://github.com/apache/flink-cdc/pull/3884#issuecomment-2608981436 Part of this change has been discussed in https://github.com/apache/flink-cdc/pull/3684#discussion_r1827701219, needs hearing from @leonardBang 👀 -- This is an automated message fr

[jira] [Updated] (FLINK-37205) Correct the state cache behavior during bump beam version

2025-01-22 Thread Dian Fu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dian Fu updated FLINK-37205: Affects Version/s: 1.20.0 1.19.0 1.18.0

[jira] [Updated] (FLINK-37205) Correct the state cache behavior during bump beam version

2025-01-22 Thread Dian Fu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dian Fu updated FLINK-37205: Affects Version/s: 1.19.1 1.18.1 1.17.2

[jira] [Updated] (FLINK-37204) Add missing StarRocks connector options for 1.2.10.

2025-01-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37204: --- Labels: pull-request-available (was: ) > Add missing StarRocks connector options for 1.2.10

[jira] [Created] (FLINK-37205) Correct the state cache behavior during bump beam version

2025-01-22 Thread Dian Fu (Jira)
Dian Fu created FLINK-37205: --- Summary: Correct the state cache behavior during bump beam version Key: FLINK-37205 URL: https://issues.apache.org/jira/browse/FLINK-37205 Project: Flink Issue Type: B

[jira] [Created] (FLINK-37204) Add missing StarRocks connector options for 1.2.10.

2025-01-22 Thread ChaomingZhang (Jira)
ChaomingZhang created FLINK-37204: - Summary: Add missing StarRocks connector options for 1.2.10. Key: FLINK-37204 URL: https://issues.apache.org/jira/browse/FLINK-37204 Project: Flink Issue T

Re: [PR] [WIP][FLINK-37148][docs] Add documents for DataStream V2 [flink]

2025-01-22 Thread via GitHub
xintongsong commented on code in PR #26057: URL: https://github.com/apache/flink/pull/26057#discussion_r1926349435 ## docs/content/docs/dev/datastream-v2/overview.md: ## @@ -0,0 +1,195 @@ +--- +title: Overview +weight: 2 +type: docs +--- + + +# Flink DataStream API Programming G

Re: [PR] [hotfix][cdc][docs] Build 3.3 docs and mark it as stable [flink-cdc]

2025-01-22 Thread via GitHub
GOODBOY008 commented on PR #3882: URL: https://github.com/apache/flink-cdc/pull/3882#issuecomment-2608962525 @ruanhang1993 PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] [FLINK-37192] [pyflink] Replace deprecated avro-python3 with avro [flink]

2025-01-22 Thread via GitHub
dianfu commented on PR #26008: URL: https://github.com/apache/flink/pull/26008#issuecomment-2608914945 @mina-asham Thanks for the update! These are test failures, could you take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] [WIP][FLINK-37148][docs] Add documents for DataStream V2 [flink]

2025-01-22 Thread via GitHub
flinkbot commented on PR #26057: URL: https://github.com/apache/flink/pull/26057#issuecomment-2608886489 ## CI report: * d7118feba7756ee0f06f9cc9e2dd87f1c2999c25 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Updated] (FLINK-37161) Cross-team verification for "Adaptive skewed join optimization for batch jobs"

2025-01-22 Thread Junrui Lee (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junrui Lee updated FLINK-37161: --- Description: In Flink 2.0, we support the capability of adaptive skewed join optimization for batch

[jira] [Updated] (FLINK-37161) Cross-team verification for "Adaptive skewed join optimization for batch jobs"

2025-01-22 Thread Junrui Lee (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junrui Lee updated FLINK-37161: --- Description: In Flink 2.0, we support the capability of adaptive skewed join optimization for batch

[jira] [Commented] (FLINK-36061) Support Apache iceberg for pipeline connector

2025-01-22 Thread ZhengYu Chen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916262#comment-17916262 ] ZhengYu Chen commented on FLINK-36061: -- I think we should implement this feature ba

[jira] [Commented] (FLINK-36838) [state] Join background threads when ForSt state backend quit

2025-01-22 Thread Zakelly Lan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916258#comment-17916258 ] Zakelly Lan commented on FLINK-36838: - follow-up merge: * master: 7b2e9c410344126a26

Re: [PR] [FLINK-37120][pipeline-connector/mysql] Add ending split chunk first to avoid TaskManager oom [flink-cdc]

2025-01-22 Thread via GitHub
beryllw commented on PR #3856: URL: https://github.com/apache/flink-cdc/pull/3856#issuecomment-2608748868 > LGTM. I'm worried if this parameter is a bit complicated, what about naming it `scan.incremental.assign-max-chunk-first.enabled` to avoid using `ending`? As `chunk` is a concept in sn

[jira] [Commented] (FLINK-36061) Support Apache iceberg for pipeline connector

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916250#comment-17916250 ] Leonard Xu commented on FLINK-36061: [~melin] +1 for your proposal, compaction optim

Re: [PR] Draft: [FLINK-36061] Support Iceberg CDC Pipeline SinkV2 [flink-cdc]

2025-01-22 Thread via GitHub
melin commented on PR #3877: URL: https://github.com/apache/flink-cdc/pull/3877#issuecomment-2608746353 Add compaction optimization? For example, commit 100 times and execute a compaction. Similar to this demos approach: https://github.com/aws-samples/iceberg-streaming-examples/blob/

[jira] [Commented] (FLINK-36061) Support Apache iceberg for pipeline connector

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916248#comment-17916248 ] Leonard Xu commented on FLINK-36061: [~rmoff]You're right, this is a duplicated issu

[jira] [Resolved] (FLINK-36061) Support Apache iceberg for pipeline connector

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu resolved FLINK-36061. Resolution: Duplicate > Support Apache iceberg for pipeline connector > --

Re: [PR] [FLINK-36698][pipeline-connector][elasticsearch] Elasticsearch Pipeline Connector support index sharding [flink-cdc]

2025-01-22 Thread via GitHub
beryllw commented on code in PR #3723: URL: https://github.com/apache/flink-cdc/pull/3723#discussion_r1926291799 ## docs/content/docs/connectors/pipeline-connectors/elasticsearch.md: ## @@ -169,7 +169,7 @@ Pipeline Connector Options optional (none) Long -

[jira] [Commented] (FLINK-35671) Support Iceberg CDC Pipeline Sink

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916246#comment-17916246 ] Leonard Xu commented on FLINK-35671: Hey [~m.orazow] , do you have time to work for

[jira] [Assigned] (FLINK-37200) Add e2e test for ElasticSearch Sink.

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-37200: -- Assignee: Yanquan Lv > Add e2e test for ElasticSearch Sink. > ---

[jira] [Assigned] (FLINK-37201) FlinkCDC Pipeline transform arithmetic functions support parameters of null and more numerical types.

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-37201: -- Assignee: Wenkai Qi > FlinkCDC Pipeline transform arithmetic functions support parameters of

[jira] [Updated] (FLINK-37148) Add documentation for datastream v2 api

2025-01-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37148: --- Labels: pull-request-available (was: ) > Add documentation for datastream v2 api >

[PR] [WIP][FLINK-37148][docs] Add documents for DataStream V2 [flink]

2025-01-22 Thread via GitHub
codenohup opened a new pull request, #26057: URL: https://github.com/apache/flink/pull/26057 ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them

[jira] [Assigned] (FLINK-37148) Add documentation for datastream v2 api

2025-01-22 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo reassigned FLINK-37148: -- Assignee: xuhuang > Add documentation for datastream v2 api > ---

Re: [PR] [FLINK-37005][table] Make StreamExecDeduplicate ouput insert only where possible [flink]

2025-01-22 Thread via GitHub
Zakelly commented on PR #26051: URL: https://github.com/apache/flink/pull/26051#issuecomment-2608722972 @pnowojski Glad to see you here. @xuyangzhong is right, you have that guarantee. The same-key records and timers will happen in order of arrival. That is, when watermark advance, the time

Re: [PR] [BP-2.0][FLINK-36838][state/forst] Fix the deadlock when quit forst state backend [flink]

2025-01-22 Thread via GitHub
Zakelly merged PR #26054: URL: https://github.com/apache/flink/pull/26054 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apach

Re: [PR] [FLINK-36838][state/forst] Fix the deadlock when quit forst state backend [flink]

2025-01-22 Thread via GitHub
Zakelly merged PR #26053: URL: https://github.com/apache/flink/pull/26053 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apach

Re: [PR] [FLINK-37005][table] Make StreamExecDeduplicate ouput insert only where possible [flink]

2025-01-22 Thread via GitHub
xuyangzhong commented on PR #26051: URL: https://github.com/apache/flink/pull/26051#issuecomment-2608696358 @pnowojski IIUC, the async state operator currently processes watermarks **only** after all prior async state requests have been completed. That means when the event time timer is tri

[jira] [Commented] (FLINK-32212) Job restarting indefinitely after an IllegalStateException from BlobLibraryCacheManager

2025-01-22 Thread Vineeth Naroju (Jira)
[ https://issues.apache.org/jira/browse/FLINK-32212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916238#comment-17916238 ] Vineeth Naroju commented on FLINK-32212: We encountered same error just now in F

Re: [PR] [FLINK-36698][pipeline-connector][elasticsearch] Elasticsearch Pipeline Connector support index sharding [flink-cdc]

2025-01-22 Thread via GitHub
beryllw commented on code in PR #3723: URL: https://github.com/apache/flink-cdc/pull/3723#discussion_r1926193705 ## docs/content/docs/connectors/pipeline-connectors/elasticsearch.md: ## @@ -169,7 +169,7 @@ Pipeline Connector Options optional (none) Long -

Re: [PR] [FLINK-36698][pipeline-connector][elasticsearch] Elasticsearch Pipeline Connector support index sharding [flink-cdc]

2025-01-22 Thread via GitHub
beryllw commented on code in PR #3723: URL: https://github.com/apache/flink-cdc/pull/3723#discussion_r1926193705 ## docs/content/docs/connectors/pipeline-connectors/elasticsearch.md: ## @@ -169,7 +169,7 @@ Pipeline Connector Options optional (none) Long -

Re: [PR] [BP-2.0][FLINK-37099][docs] Document memory/performance considerations and config possibilities for Netty4 [flink]

2025-01-22 Thread via GitHub
flinkbot commented on PR #26056: URL: https://github.com/apache/flink/pull/26056#issuecomment-2608371874 ## CI report: * ea7a81374111aed2e7869ef3e8860e7fe41146e1 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Comment Edited] (FLINK-33926) Can't start a job with a jar in the system classpath in native k8s mode

2025-01-22 Thread Trystan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916171#comment-17916171 ] Trystan edited comment on FLINK-33926 at 1/22/25 7:32 PM: -- I th

[jira] [Commented] (FLINK-33926) Can't start a job with a jar in the system classpath in native k8s mode

2025-01-22 Thread Trystan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916171#comment-17916171 ] Trystan commented on FLINK-33926: - I think it makes sense to me. Not sure how something

Re: [PR] [BP-2.0][FLINK-37183][clients] Fix symbolic links not being followed in "usrlib" [flink]

2025-01-22 Thread via GitHub
ferenc-csaky closed pull request #26052: [BP-2.0][FLINK-37183][clients] Fix symbolic links not being followed in "usrlib" URL: https://github.com/apache/flink/pull/26052 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Comment Edited] (FLINK-37183) Usrlib symlinks are not followed

2025-01-22 Thread Ferenc Csaky (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17915937#comment-17915937 ] Ferenc Csaky edited comment on FLINK-37183 at 1/22/25 6:54 PM: ---

Re: [PR] [BP-2.0][FLINK-37183][clients] Fix symbolic links not being followed in "usrlib" [flink]

2025-01-22 Thread via GitHub
ferenc-csaky commented on PR #26052: URL: https://github.com/apache/flink/pull/26052#issuecomment-2607997344 The `flink-clients` test module ran correctly, merging this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] [FLINK-37177][Doc] Update chinese documentation of json function [flink]

2025-01-22 Thread via GitHub
flinkbot commented on PR #26055: URL: https://github.com/apache/flink/pull/26055#issuecomment-2607876888 ## CI report: * 3f2569c82e8b1ba329f9a25dd86fb883ae5f11b7 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
ferenc-csaky commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607825560 > @He-Pin now that it is confirmed that the "unpooled" config solves the issue for us, how realistic do you think it is to get a Pekko release with the allocator config support inclu

Re: [PR] [FLINK-36694][Filesystems] Update Hadoop for filesystems to 3.4.1 [flink]

2025-01-22 Thread via GitHub
bbrandt closed pull request #26046: [FLINK-36694][Filesystems] Update Hadoop for filesystems to 3.4.1 URL: https://github.com/apache/flink/pull/26046 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
He-Pin commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607805457 Would you like to comment on the pr against 1.1.x too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
afedulov commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607804741 @He-Pin now that it is confirmed that the "unpooled" config solves the issue for us, how realistic do you think it is to get a Pekko release with the allocator config support included so

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
He-Pin commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607796620 I knew , after alibaba acquired, then more Java, thanks for that information. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
ferenc-csaky commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607710502 > ouch, `but Flink does not support 2.13 at all` so seems we can't drop 2.12 support very soon! We are actually trying to remove Scala from the project: https://issues.apache.

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
He-Pin commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607651794 ouch, `but Flink does not support 2.13 at all` so seems we can't drop 2.12 support very soon! -- This is an automated message from the Apache Git Service. To respond to the message, plea

[jira] [Created] (FLINK-37203) Support AltertableCommentEvent and AlterColumnCommentEvent.

2025-01-22 Thread Yanquan Lv (Jira)
Yanquan Lv created FLINK-37203: -- Summary: Support AltertableCommentEvent and AlterColumnCommentEvent. Key: FLINK-37203 URL: https://issues.apache.org/jira/browse/FLINK-37203 Project: Flink Issu

Re: [PR] [FLINK-36694][Filesystems] Update Hadoop for filesystems to 3.4.1 [flink]

2025-01-22 Thread via GitHub
bbrandt commented on PR #26046: URL: https://github.com/apache/flink/pull/26046#issuecomment-2607635603 Thanks @davidradl. Mostly using this as a temporary working area to see what the blockers are with the change in 1.20. If it high enough priority enough to move forward with, I can cher

Re: [PR] [hotfix] Fix the issue related to mounting the Logback configuration [flink]

2025-01-22 Thread via GitHub
Jam804 commented on PR #26039: URL: https://github.com/apache/flink/pull/26039#issuecomment-2607614914 I think there's nothing wrong with my code now. Thank you for your guidance. @davidradl -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
ferenc-csaky commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607584839 > @ferenc-csaky Cool But the pekko 1.2.x snapshot should be binary compatible with 1.1.x > > And there are nightly on https://repository.apache.org/content/groups/snapshots/or

[jira] [Commented] (FLINK-37177) Update chinese documentation of json function

2025-01-22 Thread qinghuan wang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916092#comment-17916092 ] qinghuan wang commented on FLINK-37177: --- hi, can you assign this one to me? [~gust

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
He-Pin commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607565950 @ferenc-csaky Cool But the pekko 1.2.x snapshot should be binary compatible with 1.1.x And there are nightly on https://repository.apache.org/content/groups/snapshots/org/apache/p

[jira] [Updated] (FLINK-37177) Update chinese documentation of json function

2025-01-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37177: --- Labels: chinese-translation pull-request-available (was: chinese-translation) > Update chi

[PR] [FLINK-37177][table] Update chinese documentation of json function [flink]

2025-01-22 Thread via GitHub
wangqinghuan opened a new pull request, #26055: URL: https://github.com/apache/flink/pull/26055 ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring th

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
ferenc-csaky commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607471427 @He-Pin That's a good point regarding cache! Probably the CI machines has less than 10 cores. I can give it a try, I just have to do a local Pekko build myself to be able to build Fl

Re: [PR] [BP-2.0][FLINK-36838][state/forst] Fix the deadlock when quit forst state backend [flink]

2025-01-22 Thread via GitHub
flinkbot commented on PR #26054: URL: https://github.com/apache/flink/pull/26054#issuecomment-2607441606 ## CI report: * 83a855a5b573f09ea590ca499b54b6410bb45478 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
He-Pin commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607379254 @ferenc-csaky I think the cache depends on how many cores you have, would you like to test the pekko 1.2.x nightly too, I would like too, but don't know how. refs: https://github.com/ap

Re: [PR] [1.20][FLINK-37100][tests] Fix `test_netty_shuffle_memory_control.sh` in CI for JDK11+ [flink]

2025-01-22 Thread via GitHub
ferenc-csaky commented on PR #25955: URL: https://github.com/apache/flink/pull/25955#issuecomment-2607368545 @afedulov So on my M4 Pro chip MacBook the test starts to run with 11MB consistentlt, and consistently fail with anything under that, since TMs cannot even start because there are no

Re: [PR] [FLINK-36973]udf dateformat support LocalZonedTimestampData datatype [flink-cdc]

2025-01-22 Thread via GitHub
hiliuxg commented on code in PR #3863: URL: https://github.com/apache/flink-cdc/pull/3863#discussion_r1925357513 ## flink-cdc-runtime/src/main/java/org/apache/flink/cdc/runtime/functions/SystemFunctionUtils.java: ## @@ -105,6 +105,13 @@ public static String dateFormat(TimestampD

[PR] [FLINK-36564][ci] Running CI in random timezone to expose more time related bugs. [flink-cdc]

2025-01-22 Thread via GitHub
lvyanquan opened a new pull request, #3650: URL: https://github.com/apache/flink-cdc/pull/3650 Refer to [this comment](https://github.com/apache/flink-cdc/pull/3648#pullrequestreview-2374769012) Copy from Paimon repo. How to verify: ![image](https://github.com/user-attachments/

[jira] [Assigned] (FLINK-37191) MySQL 'Compile and test' stage timeout after 90 minutes

2025-01-22 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-37191: -- Assignee: yux > MySQL 'Compile and test' stage timeout after 90 minutes > ---

Re: [PR] [FLINK-36838][state/forst] Fix the deadlock when quit forst state backend [flink]

2025-01-22 Thread via GitHub
flinkbot commented on PR #26053: URL: https://github.com/apache/flink/pull/26053#issuecomment-2607233199 ## CI report: * baf5b7d55de9f79a93d7356d9c4b3d4574886845 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [BP-2.0][FLINK-37183][clients] Fix symbolic links not being followed in "usrlib" [flink]

2025-01-22 Thread via GitHub
flinkbot commented on PR #26052: URL: https://github.com/apache/flink/pull/26052#issuecomment-2607231281 ## CI report: * 900a5e229422238ee0c3a8e6652b81574880d27a UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Commented] (FLINK-37202) Introduce TDengine connector

2025-01-22 Thread David Radley (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916044#comment-17916044 ] David Radley commented on FLINK-37202: -- Hi there, I think a new connector is an imp

[jira] [Created] (FLINK-37202) Introduce TDengine connector

2025-01-22 Thread kevinmen (Jira)
kevinmen created FLINK-37202: Summary: Introduce TDengine connector Key: FLINK-37202 URL: https://issues.apache.org/jira/browse/FLINK-37202 Project: Flink Issue Type: New Feature Repo

[jira] [Updated] (FLINK-37201) FlinkCDC Pipeline transform arithmetic functions support parameters of null and more numerical types.

2025-01-22 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-37201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-37201: --- Labels: pull-request-available (was: ) > FlinkCDC Pipeline transform arithmetic functions s

  1   2   >