Re: [PR] [FLINK-35667][state/forst] Implement Reducing Async State API for ForStStateBackend [flink]

2024-09-12 Thread via GitHub
Zakelly commented on code in PR #25308: URL: https://github.com/apache/flink/pull/25308#discussion_r1758282666 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/v2/InternalMergingState.java: ## @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[PR] [FLINK-35634] Add CdcUp playground CLI scripts [flink-cdc]

2024-09-12 Thread via GitHub
yuxiqian opened a new pull request, #3605: URL: https://github.com/apache/flink-cdc/pull/3605 This closes FLINK-35634. * implemented an cli utility to simplify cdc playground environment configuration * added corresponding docs in quickstart guide -- This is an automated message

Re: [PR] [FLINK-36275][rest] Remove deprecated ProgramArgsQueryParameter [flink]

2024-09-12 Thread via GitHub
flinkbot commented on PR #25323: URL: https://github.com/apache/flink/pull/25323#issuecomment-2348161085 ## CI report: * 3121e6c6fd73a5ef7b62785efb5defdd15cb9875 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-35667][state/forst] Implement Reducing Async State API for ForStStateBackend [flink]

2024-09-12 Thread via GitHub
Zakelly commented on code in PR #25308: URL: https://github.com/apache/flink/pull/25308#discussion_r1758275098 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/v2/InternalReducingState.java: ## @@ -58,6 +72,107 @@ public V get() { @Override public void a

[jira] [Updated] (FLINK-35634) Add a CDC quickstart utility

2024-09-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-35634: --- Labels: pull-request-available (was: ) > Add a CDC quickstart utility > ---

[jira] [Updated] (FLINK-36275) Remove deprecated ProgramArgsQueryParameter

2024-09-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-36275: --- Labels: pull-request-available (was: ) > Remove deprecated ProgramArgsQueryParameter >

[PR] [FLINK-36275][rest] Remove deprecated ProgramArgsQueryParameter [flink]

2024-09-12 Thread via GitHub
reswqa opened a new pull request, #25323: URL: https://github.com/apache/flink/pull/25323 ## What is the purpose of the change *ProgramArgsQueryParameter has been deprecated in Flink 1.7(see https://issues.apache.org/jira/browse/FLINK-10295). We can remove this(Breaking compatibility

[jira] [Updated] (FLINK-36275) Remove deprecated ProgramArgsQueryParameter

2024-09-12 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-36275: --- Priority: Blocker (was: Major) > Remove deprecated ProgramArgsQueryParameter >

[jira] [Updated] (FLINK-36275) Remove deprecated ProgramArgsQueryParameter

2024-09-12 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-36275: --- Parent: FLINK-3957 Issue Type: Sub-task (was: Technical Debt) > Remove deprecated ProgramAr

[jira] (FLINK-36275) Remove deprecated ProgramArgsQueryParameter

2024-09-12 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36275 ] Weijie Guo deleted comment on FLINK-36275: was (Author: weijie guo): ProgramArgsQueryParameter has been deprecated in Flink 1.7(see https://issues.apache.org/jira/browse/FLINK-10295). We can

Re: [PR] [FLINK-35667][state/forst] Implement Reducing Async State API for ForStStateBackend [flink]

2024-09-12 Thread via GitHub
fredia commented on code in PR #25308: URL: https://github.com/apache/flink/pull/25308#discussion_r1758244383 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/v2/InternalMergingState.java: ## @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Commented] (FLINK-36275) Remove deprecated ProgramArgsQueryParameter

2024-09-12 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17881474#comment-17881474 ] Weijie Guo commented on FLINK-36275: ProgramArgsQueryParameter has been deprecated i

[jira] [Updated] (FLINK-36275) Remove deprecated ProgramArgsQueryParameter

2024-09-12 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-36275: --- Description: ProgramArgsQueryParameter has been deprecated in Flink 1.7(see https://issues.apache.or

[jira] [Created] (FLINK-36275) Remove deprecated ProgramArgsQueryParameter

2024-09-12 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-36275: -- Summary: Remove deprecated ProgramArgsQueryParameter Key: FLINK-36275 URL: https://issues.apache.org/jira/browse/FLINK-36275 Project: Flink Issue Type: Technical

Re: [PR] [FLINK-36249][streaming-java] Remove RestartStrategy-related configuration getters/setters that return/set complex Java objects. [flink]

2024-09-12 Thread via GitHub
zhuzhurk commented on code in PR #25256: URL: https://github.com/apache/flink/pull/25256#discussion_r1758217831 ## flink-python/docs/reference/pyflink.datastream/stream_execution_environment.rst: ## @@ -132,4 +130,4 @@ SlotSharingGroup :toctree: api/ SlotSharingGroup

Re: [PR] [FLINK-36192][autocaler] Autocaler supports adjusting the parallelism of source vertex based on the number of partitions in Kafka or pulsars [flink-kubernetes-operator]

2024-09-12 Thread via GitHub
1996fanrui commented on code in PR #879: URL: https://github.com/apache/flink-kubernetes-operator/pull/879#discussion_r1758222419 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/JobVertexScaler.java: ## @@ -389,15 +407,61 @@ protected static int scale(

Re: [PR] [FLINK-36016] [runtime] Synchronize initialization time and clock usage in DefaultStateTransitionManager [flink]

2024-09-12 Thread via GitHub
ztison commented on PR #25319: URL: https://github.com/apache/flink/pull/25319#issuecomment-2348090458 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] [FLINK-35667][state/forst] Implement Reducing Async State API for ForStStateBackend [flink]

2024-09-12 Thread via GitHub
fredia commented on code in PR #25308: URL: https://github.com/apache/flink/pull/25308#discussion_r1758159745 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/v2/InternalReducingState.java: ## @@ -58,6 +72,108 @@ public V get() { @Override public void ad

Re: [PR] [FLINK-35667][state/forst] Implement Reducing Async State API for ForStStateBackend [flink]

2024-09-12 Thread via GitHub
Zakelly commented on code in PR #25308: URL: https://github.com/apache/flink/pull/25308#discussion_r1758160241 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/v2/InternalMergingState.java: ## @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Updated] (FLINK-36258) testRecursiveUploadForYarnS3n failed due to no AWS Credentials provided

2024-09-12 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-36258: --- Component/s: Deployment / YARN Tests (was: Build System / CI)

[jira] [Updated] (FLINK-36258) testRecursiveUploadForYarnS3n failed due to no AWS Credentials provided

2024-09-12 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-36258: --- Labels: test-stability (was: ) > testRecursiveUploadForYarnS3n failed due to no AWS Credentials pro

[jira] [Closed] (FLINK-36225) Remove deprecated methods in FLIP-382

2024-09-12 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo closed FLINK-36225. -- Resolution: Done master(2.0) via 8d4e98503fd36faa6046731d329e42cd95d5e507. > Remove deprecated method

[jira] [Closed] (FLINK-5336) Make Path immutable

2024-09-12 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-5336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo closed FLINK-5336. - Resolution: Done master(2.0) via eb19716fd1fffc5425556fbb93311daeff077682. > Make Path immutable > -

Re: [PR] [FLINK-35927][state/forst] Make ForStFlinkFileSystem write misc file in local filesystem [flink]

2024-09-12 Thread via GitHub
Zakelly commented on code in PR #25310: URL: https://github.com/apache/flink/pull/25310#discussion_r1758146251 ## flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/fs/ForStFlinkFileSystem.java: ## @@ -101,6 +156,13 @@ public boolean rename(

[jira] [Created] (FLINK-36274) Remove legacy ExternalizedCheckpointCleanup

2024-09-12 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-36274: --- Summary: Remove legacy ExternalizedCheckpointCleanup Key: FLINK-36274 URL: https://issues.apache.org/jira/browse/FLINK-36274 Project: Flink Issue Type: Sub-tas

[jira] [Commented] (FLINK-36274) Remove legacy ExternalizedCheckpointCleanup

2024-09-12 Thread Zakelly Lan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17881461#comment-17881461 ] Zakelly Lan commented on FLINK-36274: - [~spoon-lz] Would you spare some time working

Re: [PR] [FLINK-35927][state/forst] Make ForStFlinkFileSystem write misc file in local filesystem [flink]

2024-09-12 Thread via GitHub
fredia commented on code in PR #25310: URL: https://github.com/apache/flink/pull/25310#discussion_r1758136623 ## flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/fs/ForStFlinkFileSystem.java: ## @@ -125,29 +187,111 @@ public URI getUri() {

Re: [PR] [FLINK-35927][state/forst] Make ForStFlinkFileSystem write misc file in local filesystem [flink]

2024-09-12 Thread via GitHub
fredia commented on code in PR #25310: URL: https://github.com/apache/flink/pull/25310#discussion_r1758133695 ## flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/fs/ForStFlinkFileSystem.java: ## @@ -160,4 +304,16 @@ public boolean isDistri

Re: [PR] [FLINK-35927][state/forst] Make ForStFlinkFileSystem write misc file in local filesystem [flink]

2024-09-12 Thread via GitHub
fredia commented on code in PR #25310: URL: https://github.com/apache/flink/pull/25310#discussion_r1758132493 ## flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/fs/ForStFlinkFileSystem.java: ## @@ -101,6 +156,13 @@ public boolean rename(P

[jira] [Created] (FLINK-36273) Remove deprecated Table/SQL configuration in 2.0

2024-09-12 Thread xuyang (Jira)
xuyang created FLINK-36273: -- Summary: Remove deprecated Table/SQL configuration in 2.0 Key: FLINK-36273 URL: https://issues.apache.org/jira/browse/FLINK-36273 Project: Flink Issue Type: Sub-task

Re: [PR] [FLINK-36225][core] Remove deprecated method marked in FLIP-382 [flink]

2024-09-12 Thread via GitHub
reswqa merged PR #25311: URL: https://github.com/apache/flink/pull/25311 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

Re: [PR] [FLINK-5336][core] Remove IOReadableWritable from Path [flink]

2024-09-12 Thread via GitHub
reswqa merged PR #25318: URL: https://github.com/apache/flink/pull/25318 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

Re: [PR] [FLINK-35927][state/forst] Make ForStFlinkFileSystem write misc file in local filesystem [flink]

2024-09-12 Thread via GitHub
Zakelly commented on code in PR #25310: URL: https://github.com/apache/flink/pull/25310#discussion_r1758099239 ## flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/fs/ForStFlinkFileSystem.java: ## @@ -125,29 +187,111 @@ public URI getUri()

[jira] [Commented] (FLINK-35027) Implement checkpoint drain in AsyncExecutionController

2024-09-12 Thread Zakelly Lan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-35027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17881448#comment-17881448 ] Zakelly Lan commented on FLINK-35027: - Merged into master via e94d433dd0b242e68eb2eb

Re: [PR] [FLINK-35027] Drain async state request before snapshot barrier [flink]

2024-09-12 Thread via GitHub
Zakelly merged PR #25317: URL: https://github.com/apache/flink/pull/25317 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apach

Re: [PR] [hotfix][doc] MT download link to filesystem-test-utils jar [flink]

2024-09-12 Thread via GitHub
hackergin commented on PR #25315: URL: https://github.com/apache/flink/pull/25315#issuecomment-2347907388 @lsyldliu I think both need to be fixed, including master and release 1.20. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[jira] [Resolved] (FLINK-36147) Removes deprecated location field

2024-09-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl resolved FLINK-36147. --- Fix Version/s: 2.0-preview (was: 2.0.0) Resolution: Fixed mast

Re: [PR] [FLINK-36147][runtime] Removes deprecated location field [flink]

2024-09-12 Thread via GitHub
XComp merged PR #25249: URL: https://github.com/apache/flink/pull/25249 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.

[jira] [Closed] (FLINK-36239) DDB Streams Connector reprocessing due to DescribeStream inconsistencies for trimmed shards

2024-09-12 Thread Abhi Gupta (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhi Gupta closed FLINK-36239. -- > DDB Streams Connector reprocessing due to DescribeStream inconsistencies for > trimmed shards > ---

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757533937 ## flink-connector-prometheus/src/main/java/org/apache/flink/connector/prometheus/sink/http/RemoteWriteRetryStrategy.java: ## @@ -0,0 +1,129 @@ +/* + *

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757533031 ## flink-connector-prometheus/src/main/java/org/apache/flink/connector/prometheus/sink/http/RemoteWriteResponseClassifier.java: ## @@ -0,0 +1,38 @@ +/* +

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757531627 ## flink-connector-prometheus/src/main/java/org/apache/flink/connector/prometheus/sink/PrometheusSinkException.java: ## @@ -0,0 +1,33 @@ +/* + * Licensed

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757532715 ## flink-connector-prometheus/src/main/java/org/apache/flink/connector/prometheus/sink/RequestEntrySizeUtils.java: ## @@ -0,0 +1,63 @@ +/* + * Licensed

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757532179 ## flink-connector-prometheus/src/main/java/org/apache/flink/connector/prometheus/sink/PrometheusStateSerializer.java: ## @@ -0,0 +1,154 @@ +/* + * Lice

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757531446 ## flink-connector-prometheus/src/main/java/org/apache/flink/connector/prometheus/sink/PrometheusSinkConfiguration.java: ## @@ -0,0 +1,165 @@ +package or

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757530475 ## flink-connector-prometheus/src/main/java/org/apache/flink/connector/prometheus/sink/PrometheusSinkConfiguration.java: ## @@ -0,0 +1,165 @@ +package or

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757530223 ## flink-connector-prometheus/src/main/java/org/apache/flink/connector/prometheus/sink/PrometheusSinkBuilder.java: ## @@ -0,0 +1,241 @@ +/* + * Licensed

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757529466 ## flink-connector-prometheus-request-signer-amp/src/test/java/org/apache/flink/connector/prometheus/sink/aws/RequestHeaderTestUtil.java: ## @@ -0,0 +1,2

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757529253 ## flink-connector-prometheus-request-signer-amp/src/test/java/org/apache/flink/connector/prometheus/sink/aws/DummAwsSessionCredentialProvider.java: ## @

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757528729 ## flink-connector-prometheus-request-signer-amp/src/test/java/org/apache/flink/connector/prometheus/sink/aws/AWS4SignerForAMPTest.java: ## @@ -0,0 +1,73

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757527855 ## flink-connector-prometheus-request-signer-amp/src/test/java/org/apache/flink/connector/prometheus/sink/aws/RequestHeaderTestUtil.java: ## @@ -0,0 +1,2

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757526897 ## flink-connector-prometheus/README.md: ## @@ -0,0 +1,259 @@ +## Flink Prometheus connector (sink) + +Implementation of the Prometheus sink connector fo

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757525987 ## flink-connector-prometheus-request-signer-amp/src/main/java/org/apache/flink/connector/prometheus/sink/aws/AmazonManagedPrometheusWriteRequestSigner.java: ###

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757525787 ## flink-connector-prometheus-request-signer-amp/src/main/java/org/apache/flink/connector/prometheus/sink/aws/AWS4SignerForAMP.java: ## @@ -0,0 +1,347 @@

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757525575 ## flink-connector-prometheus-request-signer-amp/src/main/java/org/apache/flink/connector/prometheus/sink/aws/AWS4SignerForAMP.java: ## @@ -0,0 +1,347 @@

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1757525231 ## flink-connector-prometheus-request-signer-amp/src/test/java/org/apache/flink/connector/prometheus/sink/aws/DummAwsSessionCredentialProvider.java: ## @

Re: [PR] [FLINK-36270][Connectors/DynamoDB] Assign only children splits of a split when it is marked as finished [flink-connector-aws]

2024-09-12 Thread via GitHub
gguptp commented on PR #164: URL: https://github.com/apache/flink-connector-aws/pull/164#issuecomment-2347138069 Thanks for the review @hlteoh37! The review comments have been addressed now -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] [FLINK-35500][Connectors/DynamoDB] DynamoDb Table API Sink fails to delete elements due to key not found [flink-connector-aws]

2024-09-12 Thread via GitHub
dzikosc commented on code in PR #152: URL: https://github.com/apache/flink-connector-aws/pull/152#discussion_r1757390907 ## docs/content/docs/connectors/table/dynamodb.md: ## @@ -303,6 +303,46 @@ WITH ( ); ``` +## Specifying Primary Key For Deletes + +The DynamoDB sink supp

Re: [PR] [FLINK-35500][Connectors/DynamoDB] DynamoDb Table API Sink fails to delete elements due to key not found [flink-connector-aws]

2024-09-12 Thread via GitHub
dzikosc commented on code in PR #152: URL: https://github.com/apache/flink-connector-aws/pull/152#discussion_r1757368121 ## flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/table/DynamoDbDynamicSinkFactory.java: ## @@ -58,6 +58,17 @

Re: [PR] [FLINK-35500][Connectors/DynamoDB] DynamoDb Table API Sink fails to delete elements due to key not found [flink-connector-aws]

2024-09-12 Thread via GitHub
dzikosc commented on code in PR #152: URL: https://github.com/apache/flink-connector-aws/pull/152#discussion_r1757386924 ## flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/table/DynamoDbDynamicSinkFactory.java: ## @@ -58,6 +58,17 @

Re: [PR] [FLINK-35500][Connectors/DynamoDB] DynamoDb Table API Sink fails to delete elements due to key not found [flink-connector-aws]

2024-09-12 Thread via GitHub
dzikosc commented on code in PR #152: URL: https://github.com/apache/flink-connector-aws/pull/152#discussion_r1757386924 ## flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/table/DynamoDbDynamicSinkFactory.java: ## @@ -58,6 +58,17 @

Re: [PR] [FLINK-35500][Connectors/DynamoDB] DynamoDb Table API Sink fails to delete elements due to key not found [flink-connector-aws]

2024-09-12 Thread via GitHub
dzikosc commented on code in PR #152: URL: https://github.com/apache/flink-connector-aws/pull/152#discussion_r1757386924 ## flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/table/DynamoDbDynamicSinkFactory.java: ## @@ -58,6 +58,17 @

Re: [PR] [FLINK-35500][Connectors/DynamoDB] DynamoDb Table API Sink fails to delete elements due to key not found [flink-connector-aws]

2024-09-12 Thread via GitHub
dzikosc commented on code in PR #152: URL: https://github.com/apache/flink-connector-aws/pull/152#discussion_r1757368121 ## flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/table/DynamoDbDynamicSinkFactory.java: ## @@ -58,6 +58,17 @

Re: [PR] [FLINK-35500][Connectors/DynamoDB] DynamoDb Table API Sink fails to delete elements due to key not found [flink-connector-aws]

2024-09-12 Thread via GitHub
dzikosc commented on code in PR #152: URL: https://github.com/apache/flink-connector-aws/pull/152#discussion_r1757365286 ## flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/table/DynamoDbDynamicSink.java: ## @@ -62,7 +63,8 @@ protec

[jira] [Created] (FLINK-36272) YarnFileStageTestS3ITCase fails on master

2024-09-12 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-36272: - Summary: YarnFileStageTestS3ITCase fails on master Key: FLINK-36272 URL: https://issues.apache.org/jira/browse/FLINK-36272 Project: Flink Issue Type: Bug

[jira] [Updated] (FLINK-36272) YarnFileStageTestS3ITCase fails on master

2024-09-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl updated FLINK-36272: -- Priority: Blocker (was: Major) > YarnFileStageTestS3ITCase fails on master >

[jira] [Commented] (FLINK-36272) YarnFileStageTestS3ITCase fails on master

2024-09-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17881391#comment-17881391 ] Matthias Pohl commented on FLINK-36272: --- [~xuannan] may you have a look at this?

Re: [PR] [FLINK-34085][config] Remove deprecated string configuration keys [flink]

2024-09-12 Thread via GitHub
XComp commented on PR #25302: URL: https://github.com/apache/flink/pull/25302#issuecomment-2346944451 This PR causes a constant test failure in `master`: [YarnFileStageTestS3ITCase](https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=61979&view=logs&j=fc5181b0-e452-5c8f-68

[jira] [Resolved] (FLINK-32688) Remove deprecated exception history fields

2024-09-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-32688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl resolved FLINK-32688. --- Resolution: Fixed master: [8dc40ab70c991d67949d3498b26dd6661e816312|https://github.com/apac

Re: [PR] [FLINK-32688][runtime] Removes deprecated JobExceptionsInfo. [flink]

2024-09-12 Thread via GitHub
XComp merged PR #25248: URL: https://github.com/apache/flink/pull/25248 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.

Re: [PR] [FLINK-36147][runtime] Removes deprecated location field [flink]

2024-09-12 Thread via GitHub
XComp commented on PR #25249: URL: https://github.com/apache/flink/pull/25249#issuecomment-2346847083 Squashed commits together ([no diff](https://github.com/apache/flink/compare/9e9f00e3b9edf55bd128859729194f9d93a19433..721c6a25763e4a9936cbd1d06b95da60eabd0f36)) -- This is an automated m

[jira] [Resolved] (FLINK-36207) Disabling japicmp plugin for deprecated APIs

2024-09-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Pohl resolved FLINK-36207. --- Fix Version/s: 2.0-preview Resolution: Fixed master: [625c6774fc187aed9f875abb421dd8

Re: [PR] [FLINK-36207][build] Disables deprecated API from japicmp [flink]

2024-09-12 Thread via GitHub
XComp merged PR #25285: URL: https://github.com/apache/flink/pull/25285 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.

Re: [PR] [FLINK-36057][Connector/DynamoDB] Use Table ARN for DDB sink [flink-connector-aws]

2024-09-12 Thread via GitHub
dzikosc commented on code in PR #157: URL: https://github.com/apache/flink-connector-aws/pull/157#discussion_r1757263194 ## flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/sink/DynamoDbSinkWriter.java: ## @@ -149,7 +150,7 @@ public

Re: [PR] [FLINK-36270][Connectors/DynamoDB] Assign only children splits of a split when it is marked as finished [flink-connector-aws]

2024-09-12 Thread via GitHub
gguptp commented on code in PR #164: URL: https://github.com/apache/flink-connector-aws/pull/164#discussion_r1757216895 ## flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/source/enumerator/tracker/SplitTracker.java: ## @@ -159,12 +

Re: [PR] [FLINK-36270][Connectors/DynamoDB] Assign only children splits of a split when it is marked as finished [flink-connector-aws]

2024-09-12 Thread via GitHub
gguptp commented on code in PR #164: URL: https://github.com/apache/flink-connector-aws/pull/164#discussion_r1757216391 ## flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/source/enumerator/tracker/SplitTracker.java: ## @@ -86,17 +8

Re: [PR] [FLINK-36270][Connectors/DynamoDB] Assign only children splits of a split when it is marked as finished [flink-connector-aws]

2024-09-12 Thread via GitHub
gguptp commented on code in PR #164: URL: https://github.com/apache/flink-connector-aws/pull/164#discussion_r1757215863 ## flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/source/enumerator/DynamoDbStreamsSourceEnumerator.java: ## @

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-09-12 Thread via GitHub
nicusX commented on PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#issuecomment-2346741773 @hlteoh37 I should have addressed all comments. The configuration classes have been moved into a top-level `PrometheusSinkConfiguration` class -- This is an automated m

[jira] [Comment Edited] (FLINK-36168) AdaptiveSchedulerTest doesn't follow the production lifecycle

2024-09-12 Thread Matthias Pohl (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877742#comment-17877742 ] Matthias Pohl edited comment on FLINK-36168 at 9/12/24 3:45 PM: --

[jira] [Commented] (FLINK-36073) ApplicationMode with the K8s operator does not support downloading jars via filesystem plugins

2024-09-12 Thread Ferenc Csaky (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17881352#comment-17881352 ] Ferenc Csaky commented on FLINK-36073: -- Hey [~rmetzger], Finally had some time to

Re: [PR] [FLINK-36168][test] This is a slight improvement of the changes done in the previous FLINK-36168 commit [flink]

2024-09-12 Thread via GitHub
XComp merged PR #25305: URL: https://github.com/apache/flink/pull/25305 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.

Re: [PR] [FLINK-36001][state] Store operator name and UID in state metadata [flink]

2024-09-12 Thread via GitHub
gaborgsomogyi commented on code in PR #25267: URL: https://github.com/apache/flink/pull/25267#discussion_r1757068022 ## flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/metadata/MetadataV5Serializer.java: ## @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Softwar

Re: [PR] [FLINK-36001][state] Store operator name and UID in state metadata [flink]

2024-09-12 Thread via GitHub
gaborgsomogyi commented on code in PR #25267: URL: https://github.com/apache/flink/pull/25267#discussion_r1757059964 ## flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/metadata/MetadataV5Serializer.java: ## @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Softwar

Re: [PR] [FLINK-36192][autocaler] Autocaler supports adjusting the parallelism of source vertex based on the number of partitions in Kafka or pulsars [flink-kubernetes-operator]

2024-09-12 Thread via GitHub
mxm commented on code in PR #879: URL: https://github.com/apache/flink-kubernetes-operator/pull/879#discussion_r1757006171 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/JobVertexScaler.java: ## @@ -389,15 +413,51 @@ protected static int scale( retur

Re: [PR] [FLINK-36192][autocaler] Autocaler supports adjusting the parallelism of source vertex based on the number of partitions in Kafka or pulsars [flink-kubernetes-operator]

2024-09-12 Thread via GitHub
mxm commented on code in PR #879: URL: https://github.com/apache/flink-kubernetes-operator/pull/879#discussion_r1757005246 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/JobVertexScaler.java: ## @@ -389,15 +413,51 @@ protected static int scale( retur

Re: [PR] [FLINK-36192][autocaler] Autocaler supports adjusting the parallelism of source vertex based on the number of partitions in Kafka or pulsars [flink-kubernetes-operator]

2024-09-12 Thread via GitHub
mxm commented on code in PR #879: URL: https://github.com/apache/flink-kubernetes-operator/pull/879#discussion_r1756999617 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/JobVertexScaler.java: ## @@ -378,28 +405,70 @@ protected static int scale( // Cap

[jira] [Assigned] (FLINK-31986) Implement Integration test for FLIP-27 DataStreams API

2024-09-12 Thread Hong Liang Teoh (Jira)
[ https://issues.apache.org/jira/browse/FLINK-31986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Liang Teoh reassigned FLINK-31986: --- Assignee: Elphas Toringepi > Implement Integration test for FLIP-27 DataStreams API

Re: [PR] [FLINK-36001][state] Store operator name and UID in state metadata [flink]

2024-09-12 Thread via GitHub
gyfora commented on code in PR #25267: URL: https://github.com/apache/flink/pull/25267#discussion_r1756963082 ## flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/metadata/MetadataV5Serializer.java: ## @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] [FLINK-36192][autocaler] Autocaler supports adjusting the parallelism of source vertex based on the number of partitions in Kafka or pulsars [flink-kubernetes-operator]

2024-09-12 Thread via GitHub
mxm commented on code in PR #879: URL: https://github.com/apache/flink-kubernetes-operator/pull/879#discussion_r1756940171 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/JobVertexScaler.java: ## @@ -378,28 +405,70 @@ protected static int scale( // Cap

Re: [PR] [FLINK-36001][state] Store operator name and UID in state metadata [flink]

2024-09-12 Thread via GitHub
gaborgsomogyi commented on code in PR #25267: URL: https://github.com/apache/flink/pull/25267#discussion_r1756924303 ## flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/metadata/MetadataV5Serializer.java: ## @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Softwar

Re: [PR] [FLINK-36270][Connectors/DynamoDB] Assign only children splits of a split when it is marked as finished [flink-connector-aws]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #164: URL: https://github.com/apache/flink-connector-aws/pull/164#discussion_r1756892805 ## flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/source/enumerator/tracker/SplitTracker.java: ## @@ -159,12

Re: [PR] [FLINK-36270][Connectors/DynamoDB] Assign only children splits of a split when it is marked as finished [flink-connector-aws]

2024-09-12 Thread via GitHub
hlteoh37 commented on code in PR #164: URL: https://github.com/apache/flink-connector-aws/pull/164#discussion_r1756874586 ## flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/source/enumerator/tracker/SplitTracker.java: ## @@ -86,17

[jira] [Assigned] (FLINK-36270) DDB Streams Connector performance issue due to splitsAvailableForAssignment function

2024-09-12 Thread Hong Liang Teoh (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Liang Teoh reassigned FLINK-36270: --- Assignee: Abhi Gupta > DDB Streams Connector performance issue due to splitsAvailab

Re: [PR] [FLINK-36269][python] Remove usage about TableEnvironmentInternal#fromTableSource in python module [flink]

2024-09-12 Thread via GitHub
flinkbot commented on PR #25322: URL: https://github.com/apache/flink/pull/25322#issuecomment-2346243164 ## CI report: * 706da1369385e5e708fecb3c54a9bf5e98343161 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-36207][build] Disables deprecated API from japicmp [flink]

2024-09-12 Thread via GitHub
XComp commented on PR #25285: URL: https://github.com/apache/flink/pull/25285#issuecomment-2346176611 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [FLINK-36168][test] This is a slight improvement of the changes done in the previous FLINK-36168 commit [flink]

2024-09-12 Thread via GitHub
XComp commented on PR #25305: URL: https://github.com/apache/flink/pull/25305#issuecomment-2346171277 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [FLINK-36016] [runtime] Synchronize initialization time and clock usage in DefaultStateTransitionManager [flink]

2024-09-12 Thread via GitHub
ztison commented on PR #25319: URL: https://github.com/apache/flink/pull/25319#issuecomment-2346163318 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[jira] [Created] (FLINK-36271) Support json and jsonb type in PostgreSQL JDBC Dialect

2024-09-12 Thread Jira
Grzegorz KoĊ‚akowski created FLINK-36271: --- Summary: Support json and jsonb type in PostgreSQL JDBC Dialect Key: FLINK-36271 URL: https://issues.apache.org/jira/browse/FLINK-36271 Project: Flink

[jira] [Updated] (FLINK-36270) DDB Streams Connector performance issue due to splitsAvailableForAssignment function

2024-09-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-36270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-36270: --- Labels: pull-request-available (was: ) > DDB Streams Connector performance issue due to spl

[PR] [FLINK-36270][Connectors/DynamoDB] Assign only children splits of a split when it is marked as finished [flink-connector-aws]

2024-09-12 Thread via GitHub
gguptp opened a new pull request, #164: URL: https://github.com/apache/flink-connector-aws/pull/164 ## Purpose of the change When a split is marked as finished, we are iterating over all splits and assigning available splits to readers. On a large enough stream with en

  1   2   >