Re: [PR] [FLINK-35528][task] Skip execution of interruptible mails when yielding [flink]

2024-06-10 Thread via GitHub
rkhachatryan commented on code in PR #24904: URL: https://github.com/apache/flink/pull/24904#discussion_r1633189008 ## docs/layouts/shortcodes/generated/execution_checkpointing_configuration.html: ## Review Comment: I guess this file is a leftover and should be removed?

Re: [PR] [FLINK-35528][task] Skip execution of interruptible mails when yielding [flink]

2024-06-10 Thread via GitHub
rkhachatryan commented on code in PR #24904: URL: https://github.com/apache/flink/pull/24904#discussion_r1633189008 ## docs/layouts/shortcodes/generated/execution_checkpointing_configuration.html: ## Review Comment: I guess a leftover and should be removed? -- This is a

Re: [PR] [FLINK-35489][Autoscaler] Ensure METASPACE size is computed before the heap [flink-kubernetes-operator]

2024-06-10 Thread via GitHub
ashangit commented on PR #833: URL: https://github.com/apache/flink-kubernetes-operator/pull/833#issuecomment-2158258410 Updated the PR to take in account this [comment](https://issues.apache.org/jira/browse/FLINK-35489?focusedCommentId=17853606&page=com.atlassian.jira.plugin.system.issueta

Re: [PR] [FLINK-35525][yarn] Add a token services configuration to allow obtained token to be passed to Yarn AM [flink]

2024-06-10 Thread via GitHub
gaborgsomogyi merged PR #24891: URL: https://github.com/apache/flink/pull/24891 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [FLINK-35525][yarn] Add a token services configuration to allow obtained token to be passed to Yarn AM [flink]

2024-06-10 Thread via GitHub
gaborgsomogyi commented on PR #24891: URL: https://github.com/apache/flink/pull/24891#issuecomment-2158262913 @wForget thanks for your efforts! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[PR] [FLINK-XXX][planner] Support custom shuffle for lookup join [flink]

2024-06-10 Thread via GitHub
WencongLiu opened a new pull request, #24920: URL: https://github.com/apache/flink/pull/24920 ## What is the purpose of the change _Support custom shuffle for lookup join_ ## Verifying this change UT and ITs. ## Does this pull request potentially affect one of the

Re: [PR] [FLINK-XXX][planner] Support custom shuffle for lookup join [flink]

2024-06-10 Thread via GitHub
flinkbot commented on PR #24920: URL: https://github.com/apache/flink/pull/24920#issuecomment-2158287262 ## CI report: * 97c210a011dd34e0c19deffe746cb3e88b355ce2 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-35528][task] Skip execution of interruptible mails when yielding [flink]

2024-06-10 Thread via GitHub
pnowojski commented on code in PR #24904: URL: https://github.com/apache/flink/pull/24904#discussion_r1633225150 ## docs/layouts/shortcodes/generated/execution_checkpointing_configuration.html: ## Review Comment: fixed -- This is an automated message from the Apache Git

Re: [PR] [FLINK-32562][connectors/filesystem] Handle compressed streams correctly in ConcatFileCompactor [flink]

2024-06-10 Thread via GitHub
gaborgsomogyi commented on code in PR #24914: URL: https://github.com/apache/flink/pull/24914#discussion_r1633243846 ## flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/sink/compactor/operator/CompactService.java: ## @@ -154,23 +158,27 @@ priv

Re: [PR] [FLINK-35559][Connector/Kinesis] fix shading configuration to avoid class conflicts when using flink-connector-kinesis alongside Firehose sink [flink-connector-aws]

2024-06-10 Thread via GitHub
hlteoh37 commented on code in PR #142: URL: https://github.com/apache/flink-connector-aws/pull/142#discussion_r1633262253 ## flink-connector-aws/flink-connector-kinesis/pom.xml: ## @@ -354,6 +354,12 @@ under the License.

Re: [PR] [FLINK-35559][Connector/Kinesis] fix shading configuration to avoid class conflicts when using flink-connector-kinesis alongside Firehose sink [flink-connector-aws]

2024-06-10 Thread via GitHub
z3d1k commented on code in PR #142: URL: https://github.com/apache/flink-connector-aws/pull/142#discussion_r1633275869 ## flink-connector-aws/flink-connector-kinesis/pom.xml: ## @@ -354,6 +354,12 @@ under the License.

Re: [PR] [FLINK-32562][connectors/filesystem] Handle compressed streams correctly in ConcatFileCompactor [flink]

2024-06-10 Thread via GitHub
ferenc-csaky commented on code in PR #24914: URL: https://github.com/apache/flink/pull/24914#discussion_r1633282273 ## flink-connectors/flink-file-sink-common/src/main/java/org/apache/flink/streaming/api/functions/sink/filesystem/OutputStreamBasedPartFileWriter.java: ## @@ -50,7

Re: [PR] [Flink-35473][table] Improve Table/SQL Configuration for Flink 2.0 [flink]

2024-06-10 Thread via GitHub
lincoln-lil commented on code in PR #24889: URL: https://github.com/apache/flink/pull/24889#discussion_r164828 ## docs/layouts/shortcodes/generated/optimizer_config_configuration.html: ## @@ -65,6 +71,12 @@ Enum When it is `TRY_RESOLVE`, the optimiz

Re: [PR] [FLINK-35559][Connector/Kinesis] fix shading configuration to avoid class conflicts when using flink-connector-kinesis alongside Firehose sink [flink-connector-aws]

2024-06-10 Thread via GitHub
dannycranmer commented on PR #142: URL: https://github.com/apache/flink-connector-aws/pull/142#issuecomment-2158665814 There is a risk this will break user code depending on these libs, right? Given this is a bug, and they should not be using the shaded libs, I think this is ok, thoughts?

Re: [PR] [FLINK-35435] Add timeout Configuration to Async Sink [flink]

2024-06-10 Thread via GitHub
dannycranmer merged PR #24839: URL: https://github.com/apache/flink/pull/24839 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.

Re: [PR] [FLINK-35022][Connector/DynamoDB] Add TypeInformed DDB Element Converter as default element converter [flink-connector-aws]

2024-06-10 Thread via GitHub
dannycranmer commented on code in PR #136: URL: https://github.com/apache/flink-connector-aws/pull/136#discussion_r1633458080 ## flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/sink/DefaultDynamoDbElementConverter.java: ## @@ -0,0

Re: [PR] [FLINK-35022][Connector/DynamoDB] Add TypeInformed DDB Element Converter as default element converter [flink-connector-aws]

2024-06-10 Thread via GitHub
dannycranmer commented on PR #136: URL: https://github.com/apache/flink-connector-aws/pull/136#issuecomment-2158683163 @hlteoh37 / @z3d1k can you please take a look at this one too? -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] [FLINK-35559][Connector/Kinesis] fix shading configuration to avoid class conflicts when using flink-connector-kinesis alongside Firehose sink [flink-connector-aws]

2024-06-10 Thread via GitHub
z3d1k commented on PR #142: URL: https://github.com/apache/flink-connector-aws/pull/142#issuecomment-2158839029 > There is a risk this will break user code depending on these libs, right? Given this is a bug, and they should not be using the shaded libs, I think this is ok, thoughts?

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-06-10 Thread via GitHub
nicusX commented on code in PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#discussion_r1633781977 ## README.md: ## @@ -8,6 +8,14 @@ Apache Flink is an open source stream processing framework with powerful stream- Learn more about Flink at [https://

Re: [PR] [FLINK-33138] DataStream API implementation [flink-connector-prometheus]

2024-06-10 Thread via GitHub
nicusX commented on PR #1: URL: https://github.com/apache/flink-connector-prometheus/pull/1#issuecomment-2159216321 I moved the example application under tests and removed the separate module completely. I added links to the example application in the README. -- This is an automated m

Re: [PR] [FLINK-34470][Connectors/Kafka] Fix indefinite blocking by adjusting stopping condition in split reader [flink-connector-kafka]

2024-06-10 Thread via GitHub
morazow commented on PR #100: URL: https://github.com/apache/flink-connector-kafka/pull/100#issuecomment-2159296755 Thanks @dongwoo6kim , Tests looks good from my side 👍 (_Recently I faced similar issue which maybe related, when running batch mode with setting `startingOffset

Re: [PR] [FLINK-35371][security] Add configuration for SSL keystore and truststore type [flink]

2024-06-10 Thread via GitHub
ammar-master commented on code in PR #24919: URL: https://github.com/apache/flink/pull/24919#discussion_r1633910473 ## docs/layouts/shortcodes/generated/security_configuration.html: ## @@ -134,6 +134,12 @@ String The secret to decrypt the keystore file

Re: [PR] [FLINK-35371][security] Add configuration for SSL keystore and truststore type [flink]

2024-06-10 Thread via GitHub
ammar-master commented on code in PR #24919: URL: https://github.com/apache/flink/pull/24919#discussion_r1634004521 ## docs/layouts/shortcodes/generated/security_configuration.html: ## @@ -134,6 +134,12 @@ String The secret to decrypt the keystore file

Re: [PR] [FLINK-35489][Autoscaler] Ensure METASPACE size is computed before the heap [flink-kubernetes-operator]

2024-06-10 Thread via GitHub
1996fanrui commented on PR #833: URL: https://github.com/apache/flink-kubernetes-operator/pull/833#issuecomment-2159610745 Thanks @ashangit for the fix and everyone for the review! Merging~ -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] [FLINK-35489][Autoscaler] Ensure METASPACE size is computed before the heap [flink-kubernetes-operator]

2024-06-10 Thread via GitHub
1996fanrui merged PR #833: URL: https://github.com/apache/flink-kubernetes-operator/pull/833 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsu

Re: [PR] [FLINK-35328] AutoScale supports setting the maximum floating parallelism by the number of Pulsar partitions [flink-kubernetes-operator]

2024-06-10 Thread via GitHub
1996fanrui commented on code in PR #827: URL: https://github.com/apache/flink-kubernetes-operator/pull/827#discussion_r1634051534 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingMetricCollector.java: ## @@ -247,17 +249,36 @@ protected JobTopology getJobTopo

Re: [PR] [FLINK-35068][core][type] Introduce built-in serialization support for java.util.Set [flink]

2024-06-10 Thread via GitHub
reswqa commented on code in PR #24845: URL: https://github.com/apache/flink/pull/24845#discussion_r1634061063 ## flink-core/src/main/java/org/apache/flink/api/common/typeutils/base/SetSerializer.java: ## @@ -0,0 +1,168 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [FLINK-35068][core][type] Introduce built-in serialization support for java.util.Set [flink]

2024-06-10 Thread via GitHub
reswqa commented on code in PR #24845: URL: https://github.com/apache/flink/pull/24845#discussion_r1634076649 ## flink-core/src/main/java/org/apache/flink/api/common/typeutils/base/SetSerializer.java: ## @@ -0,0 +1,168 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [FLINK-34908][pipeline-connector][doris] Fix Mysql pipeline to doris will lost precision for timestamp [flink-cdc]

2024-06-10 Thread via GitHub
gong commented on PR #3207: URL: https://github.com/apache/flink-cdc/pull/3207#issuecomment-2159656572 > @gong would you like to open a PR for release-3.1 branch? ok, I will open the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] [pipeline-connector][doris] Fix Mysql pipeline to doris will lost precision for timestamp [flink-cdc]

2024-06-10 Thread via GitHub
gong commented on PR #3407: URL: https://github.com/apache/flink-cdc/pull/3407#issuecomment-2159661569 @leonardBang PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [FLINK-35328] AutoScale supports setting the maximum floating parallelism by the number of Pulsar partitions [flink-kubernetes-operator]

2024-06-10 Thread via GitHub
wenbingshen commented on code in PR #827: URL: https://github.com/apache/flink-kubernetes-operator/pull/827#discussion_r1634091321 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingMetricCollector.java: ## @@ -247,17 +249,36 @@ protected JobTopology getJobTop

Re: [PR] [FLINK-35531] Avoid calling hsync in flush method in BaseHadoopFsRecoverableFsDataOutputStream [flink]

2024-06-10 Thread via GitHub
cxzl25 commented on PR #24896: URL: https://github.com/apache/flink/pull/24896#issuecomment-2159691770 Gentle ping @xintongsong -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] [FLINK-35328] AutoScale supports setting the maximum floating parallelism by the number of Pulsar partitions [flink-kubernetes-operator]

2024-06-10 Thread via GitHub
wenbingshen commented on code in PR #827: URL: https://github.com/apache/flink-kubernetes-operator/pull/827#discussion_r1634118107 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingMetricCollector.java: ## @@ -247,17 +249,36 @@ protected JobTopology getJobTop

Re: [PR] [FLINK-32081][checkpoint] Compatibility between file-merging on and off across job runs [flink]

2024-06-10 Thread via GitHub
ljz2051 commented on code in PR #24873: URL: https://github.com/apache/flink/pull/24873#discussion_r1634123958 ## flink-tests/src/test/java/org/apache/flink/test/checkpointing/SnapshotFileMergingCompatibilityITCase.java: ## @@ -0,0 +1,126 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] [FLINK-35467][cdc-dist][bin] Respect externally set FLINK_CONF_DIR for CDC task configuration. [flink-cdc]

2024-06-10 Thread via GitHub
yuxiqian commented on code in PR #3398: URL: https://github.com/apache/flink-cdc/pull/3398#discussion_r1634142791 ## flink-cdc-dist/src/main/flink-cdc-bin/bin/flink-cdc.sh: ## @@ -34,11 +34,11 @@ if [[ -z $FLINK_HOME ]]; then exit 1 fi -# Setup Flink related configurations

Re: [PR] [FLINK-35371][security] Add configuration for SSL keystore and truststore type [flink]

2024-06-10 Thread via GitHub
ammar-master commented on code in PR #24919: URL: https://github.com/apache/flink/pull/24919#discussion_r1634153986 ## flink-runtime/src/test/java/org/apache/flink/runtime/net/SSLUtilsTest.java: ## @@ -306,6 +341,19 @@ void testInternalSSLWrongKeyPassword(String sslProvider) {

Re: [PR] [FLINK-35371][security] Add configuration for SSL keystore and truststore type [flink]

2024-06-10 Thread via GitHub
ammar-master commented on code in PR #24919: URL: https://github.com/apache/flink/pull/24919#discussion_r1634154163 ## flink-runtime/src/test/java/org/apache/flink/runtime/net/SSLUtilsTest.java: ## @@ -148,6 +148,17 @@ void testRESTClientSSLWrongPassword(String sslProvider) {

Re: [PR] [FLINK-35371][security] Add configuration for SSL keystore and truststore type [flink]

2024-06-10 Thread via GitHub
ammar-master commented on code in PR #24919: URL: https://github.com/apache/flink/pull/24919#discussion_r1634155084 ## flink-rpc/flink-rpc-akka/src/main/java/org/apache/flink/runtime/rpc/pekko/CustomSSLEngineProvider.java: ## @@ -59,13 +71,38 @@ public TrustManager[] trustManage

Re: [PR] [FLINK-35371][security] Add configuration for SSL keystore and truststore type [flink]

2024-06-10 Thread via GitHub
ammar-master commented on PR #24919: URL: https://github.com/apache/flink/pull/24919#issuecomment-2159778519 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] [FLINK-34108][table] Add built-in URL_ENCODE and URL_DECODE function. [flink]

2024-06-10 Thread via GitHub
liuyongvs commented on PR #24773: URL: https://github.com/apache/flink/pull/24773#issuecomment-2159808362 hi @snuyanzin @superdiaodiao do we need supports encoding args ? db2 https://www.ibm.com/docs/en/db2-for-zos/12?topic=functions-urlencode-urldecode max compute https://www.alibaba

Re: [PR] [FLINK-35531] Avoid calling hsync in flush method in BaseHadoopFsRecoverableFsDataOutputStream [flink]

2024-06-10 Thread via GitHub
xintongsong commented on PR #24896: URL: https://github.com/apache/flink/pull/24896#issuecomment-2159816793 > Thanks, I've updated. The commit message is not yet update. Have you forgot to push the changes? -- This is an automated message from the Apache Git Service. To respond

Re: [PR] [FLINK-34108][table] Add built-in URL_ENCODE and URL_DECODE function. [flink]

2024-06-10 Thread via GitHub
superdiaodiao commented on PR #24773: URL: https://github.com/apache/flink/pull/24773#issuecomment-2159830832 > hi @snuyanzin @superdiaodiao do we need supports encoding args ? > db2 https://www.ibm.com/docs/en/db2-for-zos/12?topic=functions-urlencode-urldecode > max compute https://w

Re: [PR] [FLINK-35531] Avoid calling hsync in flush method in BaseHadoopFsRecoverableFsDataOutputStream [flink]

2024-06-10 Thread via GitHub
cxzl25 commented on PR #24896: URL: https://github.com/apache/flink/pull/24896#issuecomment-2159856148 > The commit message is not yet update Sorry, it's updated now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] [FLINK-35531] Avoid calling hsync in flush method in BaseHadoopFsRecoverableFsDataOutputStream [flink]

2024-06-10 Thread via GitHub
ramkrish86 commented on code in PR #24896: URL: https://github.com/apache/flink/pull/24896#discussion_r1634237585 ## flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/BaseHadoopFsRecoverableFsDataOutputStream.java: ## @@ -59,7 +59,7 @@ public void

Re: [PR] [FLINK-35533][runtime] Support Flink hybrid shuffle integration with Apache Celeborn [flink]

2024-06-10 Thread via GitHub
reswqa commented on code in PR #24900: URL: https://github.com/apache/flink/pull/24900#discussion_r1634238153 ## flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageConsumerClient.java: ## @@ -148,12 +162,52 @@ public void

Re: [PR] [FLINK-35328] AutoScale supports setting the maximum floating parallelism by the number of Pulsar partitions [flink-kubernetes-operator]

2024-06-10 Thread via GitHub
gyfora commented on code in PR #827: URL: https://github.com/apache/flink-kubernetes-operator/pull/827#discussion_r1634247827 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingMetricCollector.java: ## @@ -247,17 +249,36 @@ protected JobTopology getJobTopology

Re: [PR] [FLINK-35328] AutoScale supports setting the maximum floating parallelism by the number of Pulsar partitions [flink-kubernetes-operator]

2024-06-10 Thread via GitHub
gyfora merged PR #827: URL: https://github.com/apache/flink-kubernetes-operator/pull/827 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr

Re: [PR] [FLINK-35068][core][type] Introduce built-in serialization support for java.util.Set [flink]

2024-06-10 Thread via GitHub
X-czh commented on PR #24845: URL: https://github.com/apache/flink/pull/24845#issuecomment-2159873214 rebased -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] [FLINK-35052] Reject unsupported versions in the webhook validator [flink-kubernetes-operator]

2024-06-10 Thread via GitHub
gyfora merged PR #831: URL: https://github.com/apache/flink-kubernetes-operator/pull/831 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr

Re: [PR] Adding affinity rule. [flink-kubernetes-operator]

2024-06-10 Thread via GitHub
gyfora closed pull request #737: Adding affinity rule. URL: https://github.com/apache/flink-kubernetes-operator/pull/737 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] Adding affinity rule. [flink-kubernetes-operator]

2024-06-10 Thread via GitHub
gyfora commented on PR #737: URL: https://github.com/apache/flink-kubernetes-operator/pull/737#issuecomment-2159881613 I am closing this PR due to inactivity, please re-open if you pick it up again -- This is an automated message from the Apache Git Service. To respond to the message, pl

Re: [PR] [FLINK-35328] AutoScale supports setting the maximum floating parallelism by the number of Pulsar partitions [flink-kubernetes-operator]

2024-06-10 Thread via GitHub
wenbingshen commented on code in PR #827: URL: https://github.com/apache/flink-kubernetes-operator/pull/827#discussion_r1634261181 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingMetricCollector.java: ## @@ -247,17 +249,36 @@ protected JobTopology getJobTop

Re: [PR] [FLINK-35531] Avoid calling hsync in flush method in BaseHadoopFsRecoverableFsDataOutputStream [flink]

2024-06-10 Thread via GitHub
xintongsong commented on PR #24896: URL: https://github.com/apache/flink/pull/24896#issuecomment-2159897218 Thanks, @cxzl25. Merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [FLINK-35531] Avoid calling hsync in flush method in BaseHadoopFsRecoverableFsDataOutputStream [flink]

2024-06-10 Thread via GitHub
xintongsong closed pull request #24896: [FLINK-35531] Avoid calling hsync in flush method in BaseHadoopFsRecoverableFsDataOutputStream URL: https://github.com/apache/flink/pull/24896 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] FLINK-35402 [flink-operator][Deployment] add labels to metadata [flink-kubernetes-operator]

2024-06-10 Thread via GitHub
gyfora merged PR #829: URL: https://github.com/apache/flink-kubernetes-operator/pull/829 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr

Re: [PR] [FLINK-20217][task] Allow certains operators to yield to unaligned checkpoint in case timers are firing [flink]

2024-06-10 Thread via GitHub
Zakelly commented on code in PR #24895: URL: https://github.com/apache/flink/pull/24895#discussion_r1634298332 ## flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/InternalTimerServiceImpl.java: ## @@ -307,18 +307,37 @@ void onProcessingTime(long time)

Re: [PR] [FLINK-20217][task] Allow certains operators to yield to unaligned checkpoint in case timers are firing [flink]

2024-06-11 Thread via GitHub
pnowojski commented on code in PR #24895: URL: https://github.com/apache/flink/pull/24895#discussion_r1634312543 ## flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/InternalTimerServiceImpl.java: ## @@ -307,18 +307,37 @@ void onProcessingTime(long time

Re: [PR] [hotfix][cdc-connector][mysql] skip SchemaChangeEvents that were not included in capturedTableFilter. [flink-cdc]

2024-06-11 Thread via GitHub
steveGuRen commented on PR #2986: URL: https://github.com/apache/flink-cdc/pull/2986#issuecomment-2159992536 @joyCurry30 @leonardBang Logically, it is considered a bug when I have to read all tables in the schema, and the first execution of "alter table sql" is not triggered. However, sinc

Re: [PR] [hotfix][cdc-connector][mysql] skip SchemaChangeEvents that were not included in capturedTableFilter. [flink-cdc]

2024-06-11 Thread via GitHub
steveGuRen commented on PR #2986: URL: https://github.com/apache/flink-cdc/pull/2986#issuecomment-216384 In the design of TiDB, I find that they take into account the scenario of having a large number of tables in a SaaS environment. Therefore, special attention is given to the performa

Re: [PR] [FLINK-32562][connectors/filesystem] Handle compressed streams correctly in ConcatFileCompactor [flink]

2024-06-11 Thread via GitHub
gaborgsomogyi commented on code in PR #24914: URL: https://github.com/apache/flink/pull/24914#discussion_r1634351465 ## flink-connectors/flink-file-sink-common/src/main/java/org/apache/flink/streaming/api/functions/sink/filesystem/OutputStreamBasedPartFileWriter.java: ## @@ -50,

Re: [PR] Add Flink 1.19.1 release [flink-web]

2024-06-11 Thread via GitHub
leonardBang commented on code in PR #745: URL: https://github.com/apache/flink-web/pull/745#discussion_r1634358337 ## docs/content/posts/2024-06-14-release-1.19.1.md: ## @@ -0,0 +1,128 @@ +--- +title: "Apache Flink 1.19.1 Release Announcement" +date: "2024-06-14T00:00:00.000Z"

Re: [PR] [FLINK-35559][Connector/Kinesis] fix shading configuration to avoid class conflicts when using flink-connector-kinesis alongside Firehose sink [flink-connector-aws]

2024-06-11 Thread via GitHub
hlteoh37 merged PR #142: URL: https://github.com/apache/flink-connector-aws/pull/142 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] [FLINK-35068][core][type] Introduce built-in serialization support for java.util.Set [flink]

2024-06-11 Thread via GitHub
reswqa merged PR #24845: URL: https://github.com/apache/flink/pull/24845 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache

Re: [PR] [FLINK-35282][FLINK-35520] PyFlink Support for Apache Beam > 2.49 [flink]

2024-06-11 Thread via GitHub
hlteoh37 commented on code in PR #24908: URL: https://github.com/apache/flink/pull/24908#discussion_r1634474656 ## docs/layouts/shortcodes/generated/python_configuration.html: ## @@ -24,7 +24,7 @@ python.executable "python" String -

Re: [PR] [FLINK-32081][checkpoint] Compatibility between file-merging on and off across job runs [flink]

2024-06-11 Thread via GitHub
Zakelly closed pull request #24873: [FLINK-32081][checkpoint] Compatibility between file-merging on and off across job runs URL: https://github.com/apache/flink/pull/24873 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] [FLINK-35540][cdc-common][cdc-connector][mysql] fix lost table when database and table are with the same name [flink-cdc]

2024-06-11 Thread via GitHub
leonardBang merged PR #3396: URL: https://github.com/apache/flink-cdc/pull/3396 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

[PR] [BP-3.1][FLINK-35545][doc] Miss 3.1.0 version in snapshot flink-cdc doc version list [flink-cdc]

2024-06-11 Thread via GitHub
GOODBOY008 opened a new pull request, #3408: URL: https://github.com/apache/flink-cdc/pull/3408 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] [BP-3.1][FLINK-35545][doc] Miss 3.1.0 version in snapshot flink-cdc doc version list [flink-cdc]

2024-06-11 Thread via GitHub
GOODBOY008 commented on PR #3408: URL: https://github.com/apache/flink-cdc/pull/3408#issuecomment-2160480202 @leonardBang PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] [BP-3.1][FLINK-35545][doc] Miss 3.1.0 version in snapshot flink-cdc doc version list [flink-cdc]

2024-06-11 Thread via GitHub
leonardBang merged PR #3408: URL: https://github.com/apache/flink-cdc/pull/3408 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [pipeline-connector][doris] Fix Mysql pipeline to doris will lost precision for timestamp [flink-cdc]

2024-06-11 Thread via GitHub
leonardBang merged PR #3407: URL: https://github.com/apache/flink-cdc/pull/3407 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [FLINK-34487][ci] Adds Python Wheels nightly GHA workflow [flink]

2024-06-11 Thread via GitHub
morazow commented on PR #24426: URL: https://github.com/apache/flink/pull/24426#issuecomment-2160556166 Thanks @HuangXingBo for the update! I have changed the GitHub actions to run the `auditwheel` repair after installing `patchelf` in Linux. Could you please have another look?

Re: [PR] [FLINK-32562][connectors/filesystem] Handle compressed streams correctly in ConcatFileCompactor [flink]

2024-06-11 Thread via GitHub
ferenc-csaky commented on code in PR #24914: URL: https://github.com/apache/flink/pull/24914#discussion_r1634722169 ## flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/sink/compactor/operator/CompactService.java: ## @@ -154,23 +158,27 @@ priva

Re: [PR] [FLINK-32562][connectors/filesystem] Handle compressed streams correctly in ConcatFileCompactor [flink]

2024-06-11 Thread via GitHub
ferenc-csaky commented on code in PR #24914: URL: https://github.com/apache/flink/pull/24914#discussion_r1634721788 ## flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/sink/compactor/operator/CompactService.java: ## @@ -154,23 +158,27 @@ priva

Re: [PR] [FLINK-32562][connectors/filesystem] Handle compressed streams correctly in ConcatFileCompactor [flink]

2024-06-11 Thread via GitHub
ferenc-csaky commented on code in PR #24914: URL: https://github.com/apache/flink/pull/24914#discussion_r1634721788 ## flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/sink/compactor/operator/CompactService.java: ## @@ -154,23 +158,27 @@ priva

Re: [PR] [FLINK-35543][HIVE] Upgrade Hive 2.3 connector to version 2.3.10 [flink]

2024-06-11 Thread via GitHub
pan3793 commented on PR #24905: URL: https://github.com/apache/flink/pull/24905#issuecomment-2160590805 ping @JingGe, is there any thing I can do before merging? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[PR] [FLINK-35540][cdc-common] Fix table missed when database and table are with the same name in release-3.1.1 [flink-cdc]

2024-06-11 Thread via GitHub
qg-lin opened a new pull request, #3409: URL: https://github.com/apache/flink-cdc/pull/3409 https://issues.apache.org/jira/browse/FLINK-35540 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] [FLINK-35022][Connector/DynamoDB] Add TypeInformed DDB Element Converter as default element converter [flink-connector-aws]

2024-06-11 Thread via GitHub
hlteoh37 commented on code in PR #136: URL: https://github.com/apache/flink-connector-aws/pull/136#discussion_r1634758095 ## flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/sink/DynamoDbTypeInformedElementConverter.java: ## @@ -0,0

Re: [PR] [FLINK-35022][Connector/DynamoDB] Add TypeInformed DDB Element Converter as default element converter [flink-connector-aws]

2024-06-11 Thread via GitHub
hlteoh37 commented on code in PR #136: URL: https://github.com/apache/flink-connector-aws/pull/136#discussion_r1634758095 ## flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/sink/DynamoDbTypeInformedElementConverter.java: ## @@ -0,0

Re: [PR] [FLINK-35022][Connector/DynamoDB] Add TypeInformed DDB Element Converter as default element converter [flink-connector-aws]

2024-06-11 Thread via GitHub
hlteoh37 commented on code in PR #136: URL: https://github.com/apache/flink-connector-aws/pull/136#discussion_r1634758095 ## flink-connector-aws/flink-connector-dynamodb/src/main/java/org/apache/flink/connector/dynamodb/sink/DynamoDbTypeInformedElementConverter.java: ## @@ -0,0

Re: [PR] [FLINK-35371][security] Add configuration for SSL keystore and truststore type [flink]

2024-06-11 Thread via GitHub
gaborgsomogyi commented on code in PR #24919: URL: https://github.com/apache/flink/pull/24919#discussion_r1634845033 ## flink-rpc/flink-rpc-akka/src/main/java/org/apache/flink/runtime/rpc/pekko/CustomSSLEngineProvider.java: ## @@ -59,13 +71,38 @@ public TrustManager[] trustManag

Re: [PR] [FLINK-35371][security] Add configuration for SSL keystore and truststore type [flink]

2024-06-11 Thread via GitHub
gaborgsomogyi commented on code in PR #24919: URL: https://github.com/apache/flink/pull/24919#discussion_r1634861389 ## flink-runtime/src/main/java/org/apache/flink/runtime/net/SSLUtils.java: ## @@ -285,7 +291,14 @@ private static KeyManagerFactory getKeyManagerFactory(

Re: [PR] [Flink-35473][table] Improve Table/SQL Configuration for Flink 2.0 [flink]

2024-06-11 Thread via GitHub
LadyForest closed pull request #24889: [Flink-35473][table] Improve Table/SQL Configuration for Flink 2.0 URL: https://github.com/apache/flink/pull/24889 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Adding Opensearch Connector v2.0.0 [flink-web]

2024-06-11 Thread via GitHub
snuyanzin merged PR #741: URL: https://github.com/apache/flink-web/pull/741 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apa

Re: [PR] Adding Opensearch Connector v1.2.0 [flink-web]

2024-06-11 Thread via GitHub
snuyanzin merged PR #740: URL: https://github.com/apache/flink-web/pull/740 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apa

Re: [PR] [hotfix] Enable ci for v1.2 [flink-connector-opensearch]

2024-06-11 Thread via GitHub
snuyanzin merged PR #47: URL: https://github.com/apache/flink-connector-opensearch/pull/47 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubs

[PR] [FLINK-35140][docs] Update Opensearch connector docs for 1.2.0 and 2.0.0 releases [flink]

2024-06-11 Thread via GitHub
snuyanzin opened a new pull request, #24921: URL: https://github.com/apache/flink/pull/24921 ## What is the purpose of the change Add docs for new Opensearch connector release In fact docs are same for both v1 and v2, so just referencing to one branch (also references to multiple b

Re: [PR] [FLINK-35140][docs] Update Opensearch connector docs for 1.2.0 and 2.0.0 releases [flink]

2024-06-11 Thread via GitHub
flinkbot commented on PR #24921: URL: https://github.com/apache/flink/pull/24921#issuecomment-2161796100 ## CI report: * 84a66fe40a34e3e74b68beee473005b903a69fe3 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-25537] [JUnit5 Migration] Module: flink-core with,Package: core [flink]

2024-06-11 Thread via GitHub
1996fanrui commented on PR #24881: URL: https://github.com/apache/flink/pull/24881#issuecomment-2161888308 > Thanks @GOODBOY008 for the quick update, LGTM now. Hi @1996fanrui, could you help double check this PR? Thank @Jiabao-Sun for the ping, I will check it these 2 days. -- This

Re: [PR] [FLINK-35540][cdc-common] Fix table missed when database and table are with the same name in release-3.1.1 [flink-cdc]

2024-06-11 Thread via GitHub
leonardBang merged PR #3409: URL: https://github.com/apache/flink-cdc/pull/3409 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink

Re: [PR] [FLINK-20217][task] Allow certains operators to yield to unaligned checkpoint in case timers are firing [flink]

2024-06-11 Thread via GitHub
1996fanrui commented on PR #24895: URL: https://github.com/apache/flink/pull/24895#issuecomment-2161973417 It seems one test related to timer fails, it maybe caused by this PR. `StreamTaskCancellationTest.testCancelTaskShouldPreventAdditionalEventTimeTimersFromBeingFired:272->testCanc

Re: [PR] [FLINK-35234][hotfix][cdc-common] Fix NullPointerException of org.apache.flink.cdc.common.configuration.ConfigurationUtils#convertToString [flink-cdc]

2024-06-11 Thread via GitHub
Jiabao-Sun commented on code in PR #3255: URL: https://github.com/apache/flink-cdc/pull/3255#discussion_r1635733171 ## flink-cdc-common/src/main/java/org/apache/flink/cdc/common/configuration/ConfigurationUtils.java: ## @@ -130,7 +130,9 @@ static Duration convertToDuration(Objec

Re: [PR] [FLINK-34545][cdc-pipeline-connector]Add OceanBase pipeline connector to Flink CDC [flink-cdc]

2024-06-11 Thread via GitHub
yuxiqian commented on PR #3360: URL: https://github.com/apache/flink-cdc/pull/3360#issuecomment-2161996630 Hi @yuanoOo, thanks for your great contribution! Since @whhe is familiar with OceanBase, do you have time to take a look? -- This is an automated message from the Apache Git Service.

[PR] [FLINK-32091][checkpoint] Add file size metrics for file-merging [flink]

2024-06-11 Thread via GitHub
fredia opened a new pull request, #24922: URL: https://github.com/apache/flink/pull/24922 ## What is the purpose of the change Add file size and file count metrics for file-merging. ## Brief change log - Add `logicalFileCount`, `logicalFileSize`, `physicalFileCoun

Re: [PR] Enable async profiler [flink-benchmarks]

2024-06-11 Thread via GitHub
Zakelly merged PR #90: URL: https://github.com/apache/flink-benchmarks/pull/90 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.

Re: [PR] [FLINK-32091][checkpoint] Add file size metrics for file-merging [flink]

2024-06-11 Thread via GitHub
flinkbot commented on PR #24922: URL: https://github.com/apache/flink/pull/24922#issuecomment-2162053937 ## CI report: * 3405d0cc2c400c9c2d3a596b600b2ebda07af2f9 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[PR] MailboxProcessor#processMailsWhenDefaultActionUnavailable avoids allocating Optional [flink]

2024-06-11 Thread via GitHub
schlosna opened a new pull request, #24923: URL: https://github.com/apache/flink/pull/24923 ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them o

[PR] [FLINK-35570] Consider PlaceholderStreamStateHandle in checkpoint file merging [flink]

2024-06-11 Thread via GitHub
Zakelly opened a new pull request, #24924: URL: https://github.com/apache/flink/pull/24924 ## What is the purpose of the change In checkpoint file merging, we should take `PlaceholderStreamStateHandle` into account during lifecycle, since it can be a file merged one. ## Brief c

[PR] Revert FLINK-34123&35068: Add built-in dedicated serialization support for Maps, Lists, Sets, and Collections [flink]

2024-06-11 Thread via GitHub
X-czh opened a new pull request, #24925: URL: https://github.com/apache/flink/pull/24925 ## What is the purpose of the change Reverting previous changes that break state compatibility. [FLINK-34123](https://issues.apache.org/jira/browse/FLINK-34123) & [FLINK-35068](https://issu

Re: [PR] MailboxProcessor#processMailsWhenDefaultActionUnavailable avoids allocating Optional [flink]

2024-06-11 Thread via GitHub
flinkbot commented on PR #24923: URL: https://github.com/apache/flink/pull/24923#issuecomment-2162103603 ## CI report: * 88f9a4cf7a8e79d861755262f84f91a4b1b0d026 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-35570] Consider PlaceholderStreamStateHandle in checkpoint file merging [flink]

2024-06-11 Thread via GitHub
flinkbot commented on PR #24924: URL: https://github.com/apache/flink/pull/24924#issuecomment-2162103785 ## CI report: * 9ce0a6f2bb299224ed170ce0944de94703253b81 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

  1   2   3   4   5   6   7   8   9   10   >