[PR] [hotfix][autoscaler] Fix that the 'deletedTotalCount' is not updated when cleaning historical event handler records in JDBC event handler [flink-kubernetes-operator]

2024-08-26 Thread via GitHub
RocMarshal opened a new pull request, #872: URL: https://github.com/apache/flink-kubernetes-operator/pull/872 ## What is the purpose of the change Fix that the 'deletedTotalCount' is not updated when cleaning historical event handler records in JDBC event handler

[PR] [hotfix] Run schema coordinator logic asynchronously to avoid blocking the main thread [flink-cdc]

2024-08-26 Thread via GitHub
yuxiqian opened a new pull request, #3577: URL: https://github.com/apache/flink-cdc/pull/3577 This tweaks `SchemaRegistry` coordinator to let it handle requests / trigger checkpoints in background threads to avoid blocking the main thread. This also adds E2e test case running on vario

Re: [PR] [FLINK-35177] Fix DataGen Connector documentation [flink]

2024-08-26 Thread via GitHub
morozov commented on code in PR #24692: URL: https://github.com/apache/flink/pull/24692#discussion_r1732153194 ## docs/content.zh/docs/connectors/datastream/datagen.md: ## @@ -65,14 +65,13 @@ Rate Limiting `Long` values at the overall source rate (across all source subtasks) no

Re: [PR] [FLINK-35177] Fix DataGen Connector documentation [flink]

2024-08-26 Thread via GitHub
morozov commented on code in PR #24692: URL: https://github.com/apache/flink/pull/24692#discussion_r1732153194 ## docs/content.zh/docs/connectors/datastream/datagen.md: ## @@ -65,14 +65,13 @@ Rate Limiting `Long` values at the overall source rate (across all source subtasks) no

Re: [PR] [FLINK-35177] Fix DataGen Connector documentation [flink]

2024-08-26 Thread via GitHub
GOODBOY008 commented on code in PR #24692: URL: https://github.com/apache/flink/pull/24692#discussion_r1732070723 ## docs/content.zh/docs/connectors/datastream/datagen.md: ## @@ -65,14 +65,13 @@ Rate Limiting `Long` values at the overall source rate (across all source subtasks)

Re: [PR] [FLINK-35177] Fix DataGen Connector documentation [flink]

2024-08-26 Thread via GitHub
GOODBOY008 commented on code in PR #24692: URL: https://github.com/apache/flink/pull/24692#discussion_r1732070723 ## docs/content.zh/docs/connectors/datastream/datagen.md: ## @@ -65,14 +65,13 @@ Rate Limiting `Long` values at the overall source rate (across all source subtasks)

Re: [PR] [FLINK-35177] Fix DataGen Connector documentation [flink]

2024-08-26 Thread via GitHub
GOODBOY008 commented on code in PR #24692: URL: https://github.com/apache/flink/pull/24692#discussion_r1732070723 ## docs/content.zh/docs/connectors/datastream/datagen.md: ## @@ -65,14 +65,13 @@ Rate Limiting `Long` values at the overall source rate (across all source subtasks)

[jira] [Commented] (FLINK-34257) Update Flink YAML Parser to Support YAML 1.2 Specification

2024-08-26 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876896#comment-17876896 ] melin commented on FLINK-34257: --- Thank you! The input format problem has been solved, it i

[jira] [Created] (FLINK-36158) Support for StreamGraph store

2024-08-26 Thread Junrui Li (Jira)
Junrui Li created FLINK-36158: - Summary: Support for StreamGraph store Key: FLINK-36158 URL: https://issues.apache.org/jira/browse/FLINK-36158 Project: Flink Issue Type: Sub-task Compon

[jira] [Updated] (FLINK-36065) Support for submitting StreamGraph

2024-08-26 Thread Junrui Li (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junrui Li updated FLINK-36065: -- Description: This ticket will encompass the following tasks: # Make StreamGraph serializable # Suppo

[jira] [Assigned] (FLINK-36065) Support for submitting StreamGraph

2024-08-26 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-36065: --- Assignee: Junrui Li > Support for submitting StreamGraph > -- > >

[jira] [Assigned] (FLINK-36063) Migrate StreamGraph and its related classes to flink-runtime

2024-08-26 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-36063: --- Assignee: Junrui Li > Migrate StreamGraph and its related classes to flink-runtime > --

[jira] [Assigned] (FLINK-36064) Refactor CollectResultIterator to eliminate dependency on OperatorID

2024-08-26 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-36064: --- Assignee: Junrui Li > Refactor CollectResultIterator to eliminate dependency on OperatorID > --

[jira] [Commented] (FLINK-34257) Update Flink YAML Parser to Support YAML 1.2 Specification

2024-08-26 Thread Junrui Li (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876888#comment-17876888 ] Junrui Li commented on FLINK-34257: --- [~melin]  I don’t see any clear connections betw

[jira] [Assigned] (FLINK-36157) FLIP-473 Introduce New SQL Operators Based on Asynchronous State APIs

2024-08-26 Thread lincoln lee (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lincoln lee reassigned FLINK-36157: --- Assignee: xuyang > FLIP-473 Introduce New SQL Operators Based on Asynchronous State APIs >

[jira] [Created] (FLINK-36157) FLIP-473 Introduce New SQL Operators Based on Asynchronous State APIs

2024-08-26 Thread xuyang (Jira)
xuyang created FLINK-36157: -- Summary: FLIP-473 Introduce New SQL Operators Based on Asynchronous State APIs Key: FLINK-36157 URL: https://issues.apache.org/jira/browse/FLINK-36157 Project: Flink Is

Re: [PR] [FLINK-34510][Runtime/State]Rename RestoreMode to RecoveryClaimMode [flink]

2024-08-26 Thread via GitHub
spoon-lz commented on code in PR #25192: URL: https://github.com/apache/flink/pull/25192#discussion_r1732036346 ## docs/layouts/shortcodes/generated/rest_v1_dispatcher.html: ## @@ -571,7 +571,7 @@ "allowNonRestoredState" : { "type" : "boolean" }, -"claimMode

[jira] [Commented] (FLINK-36130) flink cdc submit yaml task

2024-08-26 Thread zzm (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876872#comment-17876872 ] zzm commented on FLINK-36130: - [~leonard] HI > flink cdc submit yaml task > --

[jira] [Closed] (FLINK-35964) Add STARTSWITH function

2024-08-26 Thread lincoln lee (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lincoln lee closed FLINK-35964. --- Fix Version/s: 2.0.0 Resolution: Fixed Fixed in master: 4c14c763ca0f9c71d642953228b364dce3eb3

[jira] [Closed] (FLINK-35965) Add ENDSWITH function

2024-08-26 Thread lincoln lee (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lincoln lee closed FLINK-35965. --- Fix Version/s: 2.0.0 Resolution: Fixed Fixed in master: 5bbdbb2889268e879b6aa06d2cdab02fd6154

[jira] [Assigned] (FLINK-35965) Add ENDSWITH function

2024-08-26 Thread lincoln lee (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lincoln lee reassigned FLINK-35965: --- Assignee: Dylan He > Add ENDSWITH function > - > > Key:

[jira] [Assigned] (FLINK-35964) Add STARTSWITH function

2024-08-26 Thread lincoln lee (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lincoln lee reassigned FLINK-35964: --- Assignee: Dylan He > Add STARTSWITH function > --- > >

Re: [PR] [FLINK-35964][table] Add the built-in function STARTSWITH & ENDSWITH [flink]

2024-08-26 Thread via GitHub
lincoln-lil merged PR #25156: URL: https://github.com/apache/flink/pull/25156 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.a

Re: [PR] [FLINK-35467][cdc-dist][bin] Respect externally set FLINK_CONF_DIR for CDC task configuration. [flink-cdc]

2024-08-26 Thread via GitHub
github-actions[bot] commented on PR #3398: URL: https://github.com/apache/flink-cdc/pull/3398#issuecomment-2311317235 This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs. --

Re: [PR] [FLINK-34023][Connectors/Kinesis] Expose configuration of sink client retries [flink-connector-aws]

2024-08-26 Thread via GitHub
brada commented on PR #156: URL: https://github.com/apache/flink-connector-aws/pull/156#issuecomment-2311252620 It look like this PR might not be necessary if [https://github.com/apache/flink-connector-aws/pull/150](https://github.com/apache/flink-connector-aws/pull/150) gets merged. Th

[jira] [Created] (FLINK-36156) SinkV2: Provide the ability to set Committer Operator parallelism

2024-08-26 Thread Rodrigo Meneses (Jira)
Rodrigo Meneses created FLINK-36156: --- Summary: SinkV2: Provide the ability to set Committer Operator parallelism Key: FLINK-36156 URL: https://issues.apache.org/jira/browse/FLINK-36156 Project: Flin

Re: [PR] [FLINK-35177] Fix DataGen Connector documentation [flink]

2024-08-26 Thread via GitHub
morozov commented on PR #24692: URL: https://github.com/apache/flink/pull/24692#issuecomment-2310680036 I copied the rate limiting example from `DataGenerator.java` to everywhere into the documentation. I also updated `DataGenerator.java` and renamed `generatorSource` to `source` for consis

Re: [PR] [hotfix] Keep upstream pending requests in order to avoid checkpoint hanging & state inconsistency in timestamp startup mode [flink-cdc]

2024-08-26 Thread via GitHub
leonardBang merged PR #3576: URL: https://github.com/apache/flink-cdc/pull/3576 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [FLINK-35964][table] Add the built-in function STARTSWITH & ENDSWITH [flink]

2024-08-26 Thread via GitHub
lincoln-lil commented on PR #25156: URL: https://github.com/apache/flink/pull/25156#issuecomment-2310439648 The failed case `LeftOuterJoinTaskTest#testCancelLeftOuterJoinTaskWhileBuilding` in core moulde looks weird... -- This is an automated message from the Apache Git Service. To respo

Re: [PR] [FLINK-35964][table] Add the built-in function STARTSWITH & ENDSWITH [flink]

2024-08-26 Thread via GitHub
lincoln-lil commented on PR #25156: URL: https://github.com/apache/flink/pull/25156#issuecomment-2310438124 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[jira] [Closed] (FLINK-36100) Support ESCAPE in built-in function LIKE formally

2024-08-26 Thread lincoln lee (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lincoln lee closed FLINK-36100. --- Fix Version/s: 2.0.0 Resolution: Fixed Fixed in master: ae4eb7dc3ae399659e52da39f7ac74242aaca

[jira] [Updated] (FLINK-36100) Support ESCAPE in built-in function LIKE formally

2024-08-26 Thread lincoln lee (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lincoln lee updated FLINK-36100: Affects Version/s: 1.19.1 1.20.0 > Support ESCAPE in built-in function LIKE

Re: [PR] [FLINK-3154][runtime] Upgrade from Kryo v2 + Chill 0.7.6 to Kryo v5 w… [flink]

2024-08-26 Thread via GitHub
nicknezis commented on PR #22660: URL: https://github.com/apache/flink/pull/22660#issuecomment-2310424567 I added my comment on the [Jira ticket](https://issues.apache.org/jira/browse/FLINK-3154), but also posting it here for better visibility. I wanted to revisit the status of this

Re: [PR] [FLINK-36100][table] Support ESCAPE in built-in function LIKE [flink]

2024-08-26 Thread via GitHub
lincoln-lil merged PR #25225: URL: https://github.com/apache/flink/pull/25225 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.a

[jira] [Commented] (FLINK-3154) Update Kryo version from 2.24.0 to latest Kryo LTS version

2024-08-26 Thread Nick Nezis (Jira)
[ https://issues.apache.org/jira/browse/FLINK-3154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876748#comment-17876748 ] Nick Nezis commented on FLINK-3154: --- I wanted to revisit the status of this ticket. The

[jira] [Commented] (FLINK-36149) Support cleaning up expired states to prevent the continuous increase of states and add RocksDB state cleanup configuration.

2024-08-26 Thread luolei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876747#comment-17876747 ] luolei commented on FLINK-36149: Thank you for your suggestion. Yes, it is more reasonab

Re: [PR] [FLINK-34467] bump flink version to 1.20.0 [flink-connector-kafka]

2024-08-26 Thread via GitHub
HuangZhenQiu commented on PR #111: URL: https://github.com/apache/flink-connector-kafka/pull/111#issuecomment-2310366594 @AHeise Thanks. It is a very reasonable plan! Let me know anything I can help before merging PR also. -- This is an automated message from the Apache Git Service.

[jira] [Updated] (FLINK-36090) Bug with IngestDB restore operation for priority queue state in backend

2024-08-26 Thread Maxim Vershinin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Vershinin updated FLINK-36090: Description: *Summary:* Incorrect handling of priority queue states in IngestDB during re

[jira] [Updated] (FLINK-36155) ByteBuf.release() was not called before it's garbage-collected

2024-08-26 Thread Keith Lee (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keith Lee updated FLINK-36155: -- Description: The following warning logs are seen from kinesis connector indicating ``` LEAK: ByteBuf

[jira] [Updated] (FLINK-36155) ByteBuf.release() was not called before it's garbage-collected in Kinesis Connector

2024-08-26 Thread Keith Lee (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keith Lee updated FLINK-36155: -- Description: The following warning logs are seen from kinesis connector indicating memory leak. ```

[jira] [Updated] (FLINK-36155) ByteBuf.release() was not called before it's garbage-collected in Kinesis Connector

2024-08-26 Thread Keith Lee (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keith Lee updated FLINK-36155: -- Summary: ByteBuf.release() was not called before it's garbage-collected in Kinesis Connector (was: By

[jira] [Created] (FLINK-36155) ByteBuf.release() was not called before it's garbage-collected

2024-08-26 Thread Keith Lee (Jira)
Keith Lee created FLINK-36155: - Summary: ByteBuf.release() was not called before it's garbage-collected Key: FLINK-36155 URL: https://issues.apache.org/jira/browse/FLINK-36155 Project: Flink Iss

Re: [PR] [FLINK-35579] update frocksdb version to v8.10.0 [flink]

2024-08-26 Thread via GitHub
mayuehappy commented on PR #25253: URL: https://github.com/apache/flink/pull/25253#issuecomment-2310121356 `RocksDBResourceContainerTest.testGetColumnFamilyOptionsWithPartitionedIndex` failed because RocksDB-V8.10.0 Overrided the `equals()` of `org.rocksdb.BloomFilter` .In RocksDBResourceC

Re: [PR] [FLINK-35773][docs] Document s5cmd [flink]

2024-08-26 Thread via GitHub
rkhachatryan commented on code in PR #25235: URL: https://github.com/apache/flink/pull/25235#discussion_r1731164817 ## docs/content/docs/deployment/filesystems/s3.md: ## @@ -164,4 +164,38 @@ The `s3.entropy.key` defines the string in paths that is replaced by the random If a f

Re: [PR] [FLINK-35579] update frocksdb version to v8.10.0 [flink]

2024-08-26 Thread via GitHub
mayuehappy commented on PR #25253: URL: https://github.com/apache/flink/pull/25253#issuecomment-2310071852 > Is the CI failure related with compilation or release of the frocksdb? @Zakelly The compilation looks fine There are two main reasons why CI has failed The error message f

[jira] [Commented] (FLINK-34257) Update Flink YAML Parser to Support YAML 1.2 Specification

2024-08-26 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876695#comment-17876695 ] melin commented on FLINK-34257: --- 添加  -Dkubernetes.container.image.pull-secrets=docker-reg-

[jira] [Updated] (FLINK-34257) Update Flink YAML Parser to Support YAML 1.2 Specification

2024-08-26 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated FLINK-34257: -- Attachment: image-2024-08-26-19-55-06-927.png > Update Flink YAML Parser to Support YAML 1.2 Specification >

[jira] [Updated] (FLINK-34257) Update Flink YAML Parser to Support YAML 1.2 Specification

2024-08-26 Thread melin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-34257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] melin updated FLINK-34257: -- Attachment: image-2024-08-26-19-53-40-349.png > Update Flink YAML Parser to Support YAML 1.2 Specification >

[jira] [Closed] (FLINK-36140) Log a warning when pods are terminated by kubernetes

2024-08-26 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi closed FLINK-36140. - > Log a warning when pods are terminated by kubernetes > ---

[jira] [Resolved] (FLINK-36140) Log a warning when pods are terminated by kubernetes

2024-08-26 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi resolved FLINK-36140. --- Fix Version/s: 2.0.0 Resolution: Fixed 9bcd8f4 on master > Log a warning when pods a

Re: [PR] [FLINK-36140] Log a warning when pods are terminated by kubernetes [flink]

2024-08-26 Thread via GitHub
gaborgsomogyi merged PR #25242: URL: https://github.com/apache/flink/pull/25242 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [FLINK-36117] Implement AsyncKeyedStateBackend for RocksDBKeyedStateBackend and HeapKeyedStateBackend [flink]

2024-08-26 Thread via GitHub
wqshr12345 commented on code in PR #25233: URL: https://github.com/apache/flink/pull/25233#discussion_r1731118174 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/heap/HeapKeyedStateBackend.java: ## @@ -475,6 +482,32 @@ public LocalRecoveryConfig getLocalRecoveryCon

Re: [PR] [FLINK-36117] Implement AsyncKeyedStateBackend for RocksDBKeyedStateBackend and HeapKeyedStateBackend [flink]

2024-08-26 Thread via GitHub
wqshr12345 commented on code in PR #25233: URL: https://github.com/apache/flink/pull/25233#discussion_r1731117894 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/v2/ValueStateWrapper.java: ## @@ -0,0 +1,87 @@ +// +// Source code recreated from a .class file by Inte

Re: [PR] [FLINK-35964][table] Add the built-in function STARTSWITH & ENDSWITH [flink]

2024-08-26 Thread via GitHub
dylanhz commented on PR #25156: URL: https://github.com/apache/flink/pull/25156#issuecomment-2309963258 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [FLINK-34510][Runtime/State]Rename RestoreMode to RecoveryClaimMode [flink]

2024-08-26 Thread via GitHub
Zakelly commented on code in PR #25192: URL: https://github.com/apache/flink/pull/25192#discussion_r1731064677 ## docs/layouts/shortcodes/generated/rest_v1_dispatcher.html: ## @@ -571,7 +571,7 @@ "allowNonRestoredState" : { "type" : "boolean" }, -"claimMode"

Re: [PR] [FLINK-36149][table]In Rank operators, add the cleanupInRocksdbCompactFilter(queryTimeAfterNumEntries) parameter settin [flink]

2024-08-26 Thread via GitHub
lexluo09 commented on PR #25257: URL: https://github.com/apache/flink/pull/25257#issuecomment-2309895864 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[jira] [Commented] (FLINK-36149) Support cleaning up expired states to prevent the continuous increase of states and add RocksDB state cleanup configuration.

2024-08-26 Thread lincoln lee (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876663#comment-17876663 ] lincoln lee commented on FLINK-36149: - [~lexluo09] Thanks for reporting this! It see

Re: [PR] [FLINK-36112][Connector/Filesystem].Add Support for CreateFlag.NO_LOCAL_WRITE in FLINK on YARN's File Creation to Manage Disk Space and Network Load in Labeled YARN Nodes [flink]

2024-08-26 Thread via GitHub
liangyu-1 commented on PR #25226: URL: https://github.com/apache/flink/pull/25226#issuecomment-2309843416 @xintongsong Hi, would you please help me check this issue? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [PR] [FLINK-36117] Implement AsyncKeyedStateBackend for RocksDBKeyedStateBackend and HeapKeyedStateBackend [flink]

2024-08-26 Thread via GitHub
wqshr12345 commented on code in PR #25233: URL: https://github.com/apache/flink/pull/25233#discussion_r1731019339 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/v2/ValueStateWrapper.java: ## @@ -0,0 +1,87 @@ +// +// Source code recreated from a .class file by Inte

Re: [PR] [FLINK-36117] Implement AsyncKeyedStateBackend for RocksDBKeyedStateBackend and HeapKeyedStateBackend [flink]

2024-08-26 Thread via GitHub
wqshr12345 commented on code in PR #25233: URL: https://github.com/apache/flink/pull/25233#discussion_r1731017991 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/v2/StateDescriptorTransformer.java: ## @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundat

[jira] [Commented] (FLINK-36128) The in-flight data will meet uncoverable issue when restore from checkpoint once old schema has been flushed to downstream systems

2024-08-26 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876650#comment-17876650 ] Leonard Xu commented on FLINK-36128: Thanks [~xiqian_yu] for the reporting, I adjust

[jira] [Updated] (FLINK-36128) The in-flight data will meet unrecoverable issue when restore from checkpoint once old schema has been flushed to downstream systems

2024-08-26 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-36128: --- Summary: The in-flight data will meet unrecoverable issue when restore from checkpoint once old sche

Re: [PR] [FLINK-35579] update frocksdb version to v8.10.0 [flink]

2024-08-26 Thread via GitHub
Zakelly commented on PR #25253: URL: https://github.com/apache/flink/pull/25253#issuecomment-2309808133 Is the CI failure related with compilation or release of the frocksdb? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Updated] (FLINK-36128) The in-flight data will meet uncoverable issue when restore from checkpoint once old schema has been flushed to downstream systems

2024-08-26 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-36128: --- Summary: The in-flight data will meet uncoverable issue when restore from checkpoint once old schema

[jira] [Updated] (FLINK-36128) The in-flight data will meet uncoverable issue once old schema flushed to downstream systems

2024-08-26 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-36128: --- Summary: The in-flight data will meet uncoverable issue once old schema flushed to downstream system

[jira] [Updated] (FLINK-36128) Promote LENIENT mode as the default schema evolution behavior

2024-08-26 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-36128: --- Affects Version/s: cdc-3.1.1 > Promote LENIENT mode as the default schema evolution behavior > -

[jira] [Updated] (FLINK-36128) Promote LENIENT mode as the default schema evolution behavior

2024-08-26 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-36128: --- Fix Version/s: cdc-3.2.0 > Promote LENIENT mode as the default schema evolution behavior > -

[jira] [Assigned] (FLINK-36128) Promote LENIENT mode as the default schema evolution behavior

2024-08-26 Thread Leonard Xu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-36128: -- Assignee: yux > Promote LENIENT mode as the default schema evolution behavior > -

[jira] [Updated] (FLINK-36154) Support deserialize json ignore field key case

2024-08-26 Thread KevinyhZou (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KevinyhZou updated FLINK-36154: --- Affects Version/s: 1.20.0 > Support deserialize json ignore field key case > ---

Re: [PR] [FLINK-36128] Promote LENIENT as the default schema change behavior [flink-cdc]

2024-08-26 Thread via GitHub
leonardBang merged PR #3574: URL: https://github.com/apache/flink-cdc/pull/3574 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [FLINK-36149][table]In Rank operators, add the cleanupInRocksdbCompactFilter(queryTimeAfterNumEntries) parameter settin [flink]

2024-08-26 Thread via GitHub
lexluo09 commented on PR #25257: URL: https://github.com/apache/flink/pull/25257#issuecomment-2309709712 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] [FLINK-35414] Rework last-state upgrade mode to support job cancellation as suspend mechanism [flink-kubernetes-operator]

2024-08-26 Thread via GitHub
gyfora opened a new pull request, #871: URL: https://github.com/apache/flink-kubernetes-operator/pull/871 ## What is the purpose of the change Rework the last-state upgrade mode to not be solely reliant on HA metadata but to be flexible and use the job cancel mechanism in other cases.

[jira] [Updated] (FLINK-35414) Cancel jobs through rest api for last-state upgrades

2024-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-35414: --- Labels: pull-request-available (was: ) > Cancel jobs through rest api for last-state upgrad

Re: [PR] [FLINK-36100][table] Support ESCAPE in built-in function LIKE [flink]

2024-08-26 Thread via GitHub
dylanhz commented on PR #25225: URL: https://github.com/apache/flink/pull/25225#issuecomment-2309692044 > Also verified the result with escaping characters, all results with supported patterns are consistent with mysql: ![image](https://private-user-images.githubusercontent.com/3712895/3612

[jira] [Updated] (FLINK-36154) Support deserialize json ignore field key case

2024-08-26 Thread KevinyhZou (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KevinyhZou updated FLINK-36154: --- Summary: Support deserialize json ignore field key case (was: Support parse json ignore field key c

Re: [PR] [FLINK-36149][table]In Rank operators, add the cleanupInRocksdbCompactFilter(queryTimeAfterNumEntries) parameter settin [flink]

2024-08-26 Thread via GitHub
flinkbot commented on PR #25257: URL: https://github.com/apache/flink/pull/25257#issuecomment-2309673730 ## CI report: * eeb478f86e8d8a9779fa35762a18f27a8769d7a6 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-33678][streaming-java] Remove configuration getters/setters that return/set complex Java objects. [flink]

2024-08-26 Thread via GitHub
flinkbot commented on PR #25256: URL: https://github.com/apache/flink/pull/25256#issuecomment-2309673256 ## CI report: * c168fa357dfba154a6ecd847ec03a20b977c738b UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Updated] (FLINK-36149) Support cleaning up expired states to prevent the continuous increase of states and add RocksDB state cleanup configuration.

2024-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-36149: --- Labels: pull-request-available (was: ) > Support cleaning up expired states to prevent the

[PR] [FLINK-36149][table]In Rank operators, add the cleanupInRocksdbCompactFilter(queryTimeAfterNumEntries) parameter settin [flink]

2024-08-26 Thread via GitHub
lexluo09 opened a new pull request, #25257: URL: https://github.com/apache/flink/pull/25257 ## What is the purpose of the change *In Rank operators, add the cleanupInRocksdbCompactFilter(queryTimeAfterNumEntries) parameter setting, allowing adjustment based on the size of use

Re: [PR] [FLINK-36154][formats] Support parse json ignore field key case [flink]

2024-08-26 Thread via GitHub
flinkbot commented on PR #25255: URL: https://github.com/apache/flink/pull/25255#issuecomment-2309660974 ## CI report: * cda95febf414e947f12df59608f0cedbe065c21e UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Updated] (FLINK-33678) Remove configuration getters/setters that return/set complex Java objects

2024-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-33678: --- Labels: 2.0-related pull-request-available (was: 2.0-related) > Remove configuration getter

Re: [PR] [FLINK-36150] tables.exclude is still valid if scan.binlog.newly-added-table.enabled is true. [flink-cdc]

2024-08-26 Thread via GitHub
loserwang1024 commented on PR #3573: URL: https://github.com/apache/flink-cdc/pull/3573#issuecomment-2309661159 > Could you please add some IT tests for this in the mysql source cdc connector? I have done it in org.apache.flink.cdc.connectors.mysql.source.MysqlPipelineNewlyAddedTable

[PR] [FLINK-33678][streaming-java] Remove configuration getters/setters that return/set complex Java objects. [flink]

2024-08-26 Thread via GitHub
JunRuiLee opened a new pull request, #25256: URL: https://github.com/apache/flink/pull/25256 ## What is the purpose of the change [FLINK-33581](https://issues.apache.org/jira/browse/FLINK-33581)/FLIP-381 Deprecate configuration getters/setters that return/set complex J

[PR] [FLINK-36154][formats] Support parse json ignore field key case [flink]

2024-08-26 Thread via GitHub
KevinyhZou opened a new pull request, #25255: URL: https://github.com/apache/flink/pull/25255 ## What is the purpose of the change Support parse json string ignore the field key case. ## Brief change log Introduce a config `json.ignore-key-case` to ignore the key case

[jira] [Updated] (FLINK-36154) Support parse json ignore field key case

2024-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-36154: --- Labels: pull-request-available (was: ) > Support parse json ignore field key case > ---

[jira] [Created] (FLINK-36154) Support parse json ignore field key case

2024-08-26 Thread KevinyhZou (Jira)
KevinyhZou created FLINK-36154: -- Summary: Support parse json ignore field key case Key: FLINK-36154 URL: https://issues.apache.org/jira/browse/FLINK-36154 Project: Flink Issue Type: Improvement

[jira] [Closed] (FLINK-35772) Deprecate/remove DuplicatingFileSystem

2024-08-26 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Nowojski closed FLINK-35772. -- Fix Version/s: 2.0.0 Resolution: Fixed merged commit 56c8199 into apache:master > Depr

[jira] [Closed] (FLINK-36108) Wait for state download on cancellation to enforce cleanup

2024-08-26 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Nowojski closed FLINK-36108. -- Fix Version/s: 2.0.0 Resolution: Fixed Merged as 576b30820ff to master > Wait for stat

[jira] [Commented] (FLINK-36145) Change JobSpec.flinkStateSnapshotReference to snapshotReference

2024-08-26 Thread Gyula Fora (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876624#comment-17876624 ] Gyula Fora commented on FLINK-36145: I also like initialStateReference more, we coul

[jira] [Closed] (FLINK-35771) Limit s5cmd resource usage

2024-08-26 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Nowojski closed FLINK-35771. -- Fix Version/s: 2.0.0 Resolution: Fixed merged as 0daca7b6db8 to master > Limit s5cmd r

[jira] [Comment Edited] (FLINK-35770) Interrupt s5cmd call on cancellation

2024-08-26 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876622#comment-17876622 ] Piotr Nowojski edited comment on FLINK-35770 at 8/26/24 8:16 AM: -

[jira] [Closed] (FLINK-35770) Interrupt s5cmd call on cancellation

2024-08-26 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Nowojski closed FLINK-35770. -- Fix Version/s: 2.0.0 Resolution: Fixed merged as 0daca7b6db8 to master > Interrupt s5c

Re: [PR] [FLINK-35770][FLINK-35771][FLINK-36108] Limit resource usage, interrupt and better cleanup s5cmd [flink]

2024-08-26 Thread via GitHub
pnowojski merged PR #25228: URL: https://github.com/apache/flink/pull/25228 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apa

[jira] [Updated] (FLINK-30629) ClientHeartbeatTest.testJobRunningIfClientReportHeartbeat is unstable

2024-08-26 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-30629: --- Affects Version/s: 1.19.1 > ClientHeartbeatTest.testJobRunningIfClientReportHeartbeat is unstable >

[jira] [Commented] (FLINK-30629) ClientHeartbeatTest.testJobRunningIfClientReportHeartbeat is unstable

2024-08-26 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-30629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17876620#comment-17876620 ] Weijie Guo commented on FLINK-30629: 1.19: https://dev.azure.com/apache-flink/apach

[jira] [Updated] (FLINK-36149) Support cleaning up expired states to prevent the continuous increase of states and add RocksDB state cleanup configuration.

2024-08-26 Thread luolei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] luolei updated FLINK-36149: --- Description: 1、Problem description: {code:java} select * from  (      SELECT *, ROW_NUMBER() OVER (PARTITION

[jira] [Updated] (FLINK-36149) Support cleaning up expired states to prevent the continuous increase of states and add RocksDB state cleanup configuration.

2024-08-26 Thread luolei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] luolei updated FLINK-36149: --- Summary: Support cleaning up expired states to prevent the continuous increase of states and add RocksDB sta

Re: [PR] [FLINK-34467] bump flink version to 1.20.0 [flink-connector-kafka]

2024-08-26 Thread via GitHub
AHeise commented on PR #111: URL: https://github.com/apache/flink-connector-kafka/pull/111#issuecomment-2309529736 > Thanks for the reply. I totally understand the pain points of maintain multiple flink version compatibility for a connector. In each Flink release, there are always some new