[jira] [Updated] (FLINK-33935) Improve the default value doc and logic for some state backend and checkpoint related options

2023-12-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-33935: --- Labels: pull-request-available (was: ) > Improve the default value doc and logic for some s

[PR] [FLINK-33935][checkpoint] Improve the default value logic related to `execution.checkpointing.tolerable-failed-checkpoints` and `state.backend.type` [flink]

2023-12-24 Thread via GitHub
1996fanrui opened a new pull request, #23987: URL: https://github.com/apache/flink/pull/23987 ## What is the purpose of the change Some state backend and checkpoint related options don't set the default value directly, but but they implement default value based on code. Such as:

[jira] [Commented] (FLINK-33932) Support retry mechanism for rocksdb uploader

2023-12-24 Thread xiangyu feng (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17800258#comment-17800258 ] xiangyu feng commented on FLINK-33932: -- Hi [~dianer17], thanks for creating  this.

[jira] [Updated] (FLINK-33935) Improve the default value doc and logic for some state backend and checkpoint related options

2023-12-24 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Fan updated FLINK-33935: Description: Some state backend and checkpoint related options don't set the default value directly, but

[jira] [Created] (FLINK-33935) Improve the default value doc and logic for some state backend and checkpoint related options

2023-12-24 Thread Rui Fan (Jira)
Rui Fan created FLINK-33935: --- Summary: Improve the default value doc and logic for some state backend and checkpoint related options Key: FLINK-33935 URL: https://issues.apache.org/jira/browse/FLINK-33935 P

Re: [PR] [FLINK-33932][checkpointing] Add retry mechanism in RocksDBStateUploader [flink]

2023-12-24 Thread via GitHub
flinkbot commented on PR #23986: URL: https://github.com/apache/flink/pull/23986#issuecomment-1868812174 ## CI report: * 4f3d0851c216a7178ac673ca72b6b6fd031baf49 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Updated] (FLINK-33932) Support retry mechanism for rocksdb uploader

2023-12-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-33932: --- Labels: pull-request-available (was: ) > Support retry mechanism for rocksdb uploader > ---

[PR] [FLINK-33932][checkpointing] Add retry mechanism in RocksDBStateUploader [flink]

2023-12-24 Thread via GitHub
xiangyuf opened a new pull request, #23986: URL: https://github.com/apache/flink/pull/23986 ## What is the purpose of the change Add retry mechanism in RocksDBStateUploader ## Brief change log - Introduce RetryStrategy in RocksDBStateUploader when uploadFilesToChec

[jira] [Updated] (FLINK-33934) Flink SQL Source use raw format maybe lead to data lost

2023-12-24 Thread Cai Liuyang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-33934: Description: In our product we encounter a case that lead to data lost, the job info:     1. using

[jira] [Updated] (FLINK-33934) Flink SQL Source use raw format maybe lead to data lost

2023-12-24 Thread Cai Liuyang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-33934: Description: In our product we encounter a case that lead to data lost, the job info:     1. using

[jira] [Updated] (FLINK-33934) Flink SQL Source use raw format maybe lead to data lost

2023-12-24 Thread Cai Liuyang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-33934: Description: In our product we encounter a case that lead to data lost, the job info:     1. using

[jira] [Updated] (FLINK-33934) Flink SQL Source use raw format maybe lead to data lost

2023-12-24 Thread Cai Liuyang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cai Liuyang updated FLINK-33934: Description: In our product we encounter a case that lead to data lost, the job info:     1. using

[jira] [Created] (FLINK-33934) Flink SQL Source use raw format maybe lead to data lost

2023-12-24 Thread Cai Liuyang (Jira)
Cai Liuyang created FLINK-33934: --- Summary: Flink SQL Source use raw format maybe lead to data lost Key: FLINK-33934 URL: https://issues.apache.org/jira/browse/FLINK-33934 Project: Flink Issue T

[jira] [Comment Edited] (FLINK-33681) Display source/sink numRecordsIn/Out & numBytesIn/Out on UI

2023-12-24 Thread Zhanghao Chen (Jira)
gt; Components: Runtime / Metrics >Affects Versions: 1.18.0, 1.17.2 >Reporter: Zhanghao Chen >Priority: Major > Attachments: screenshot-20231224-195605-1.png, > screenshot-20231225-120421.png > > > Currently, the numRecordsIn &

[jira] [Updated] (FLINK-33865) exponential-delay.attempts-before-reset-backoff doesn't work when it's set in Job Configuration

2023-12-24 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Fan updated FLINK-33865: Parent: FLINK-33735 Issue Type: Sub-task (was: Bug) > exponential-delay.attempts-before-reset-bac

[jira] [Updated] (FLINK-33681) Display source/sink numRecordsIn/Out & numBytesIn/Out on UI

2023-12-24 Thread Zhanghao Chen (Jira)
omponents: Runtime / Metrics >Affects Versions: 1.18.0, 1.17.2 >Reporter: Zhanghao Chen >Priority: Major > Attachments: screenshot-20231224-195605-1.png, > screenshot-20231225-120421.png > > > Currently, the numRecordsIn & numBytesIn me

[jira] [Updated] (FLINK-33681) Display source/sink numRecordsIn/Out & numBytesIn/Out on UI

2023-12-24 Thread Zhanghao Chen (Jira)
ntime / Metrics >Affects Versions: 1.18.0, 1.17.2 >Reporter: Zhanghao Chen >Priority: Major > Attachments: screenshot-20231224-195605-1.png, > screenshot-20231225-120421-1.png, screenshot-20231225-120421.png > > > Currently, the n

[jira] [Commented] (FLINK-33681) Display source/sink numRecordsIn/Out & numBytesIn/Out on UI

2023-12-24 Thread Zhanghao Chen (Jira)
1 > Project: Flink > Issue Type: Bug > Components: Runtime / Metrics >Affects Versions: 1.18.0, 1.17.2 >Reporter: Zhanghao Chen >Priority: Major > Attachments: screenshot-20231224-195605-1.png, > screenshot-20

[jira] [Updated] (FLINK-33681) Display source/sink numRecordsIn/Out & numBytesIn/Out on UI

2023-12-24 Thread Zhanghao Chen (Jira)
ntime / Metrics >Affects Versions: 1.18.0, 1.17.2 >Reporter: Zhanghao Chen >Priority: Major > Attachments: screenshot-20231224-195605-1.png, > screenshot-20231225-120421.png > > > Currently, the numRecordsIn & numBytesIn metrics for sources

Re: [PR] [FLINK-33743][runtime] Support consuming multiple subpartitions on a single channel [flink]

2023-12-24 Thread via GitHub
yunfengzhou-hub commented on code in PR #23927: URL: https://github.com/apache/flink/pull/23927#discussion_r1435935387 ## flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannelTest.java: ## @@ -1736,7 +1738,8 @@ private RemoteInputC

[jira] [Commented] (FLINK-33681) Display source/sink numRecordsIn/Out & numBytesIn/Out on UI

2023-12-24 Thread Zhanghao Chen (Jira)
ink > Issue Type: Bug > Components: Runtime / Metrics >Affects Versions: 1.18.0, 1.17.2 >Reporter: Zhanghao Chen >Priority: Major > Attachments: screenshot-20231224-195605-1.png > > > Currently, the numRecordsIn & numBytes

[jira] [Updated] (FLINK-33681) Display source/sink numRecordsIn/Out & numBytesIn/Out on UI

2023-12-24 Thread Zhanghao Chen (Jira)
cs. As a result, even though new connectors supporting FLIP-33 metrics will report operator-level I/O metrics, we still cannot see the metrics on dashboard. This ticket serves as an umbrella issue to integrate standard source/sink I/O metrics with the corresponding task I/O metrics.  !screenshot-

[jira] [Updated] (FLINK-33681) Display source/sink numRecordsIn/Out & numBytesIn/Out on UI

2023-12-24 Thread Zhanghao Chen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhanghao Chen updated FLINK-33681: -- Attachment: screenshot-20231224-195605-1.png Description: Currently, the numRecordsIn

[jira] [Updated] (FLINK-33681) Display source/sink numRecordsIn/Out & numBytesIn/Out on UI

2023-12-24 Thread Zhanghao Chen (Jira)
omponents: Runtime / Metrics >Affects Versions: 1.18.0, 1.17.2 >Reporter: Zhanghao Chen >Priority: Major > Attachments: screenshot-20231224-195605-1.png > > > Currently, the numRecordsIn & numBytesIn metrics for sources and the > numRecord

[jira] [Updated] (FLINK-33933) SerializedThrowable will be java.lang.StackOverflowError when AsyncLookupFunction throw an exception

2023-12-24 Thread KarlManong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KarlManong updated FLINK-33933: --- Description: Here is a simple example {code:java} // example public class TableA implements LookupT

[jira] [Commented] (FLINK-33933) SerializedThrowable will be java.lang.StackOverflowError when AsyncLookupFunction throw an exception

2023-12-24 Thread KarlManong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17800230#comment-17800230 ] KarlManong commented on FLINK-33933: `SerializedThrowable` method `addAllSuppressed`

[jira] [Created] (FLINK-33933) SerializedThrowable will be java.lang.StackOverflowError when AsyncLookupFunction throw an exception

2023-12-24 Thread KarlManong (Jira)
KarlManong created FLINK-33933: -- Summary: SerializedThrowable will be java.lang.StackOverflowError when AsyncLookupFunction throw an exception Key: FLINK-33933 URL: https://issues.apache.org/jira/browse/FLINK-33933

[jira] [Created] (FLINK-33932) Support retry mechanism for rocksdb uploader

2023-12-24 Thread Guojun Li (Jira)
Guojun Li created FLINK-33932: - Summary: Support retry mechanism for rocksdb uploader Key: FLINK-33932 URL: https://issues.apache.org/jira/browse/FLINK-33932 Project: Flink Issue Type: Improvemen

Re: [PR] [FLINK-33792] Generate the same code for the same logic [flink]

2023-12-24 Thread via GitHub
zoudan commented on PR #23984: URL: https://github.com/apache/flink/pull/23984#issuecomment-1868679111 @libenchao Thanks for reviewing, I have remove the configuration, please take a look when you have time. -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] [FLINK-33743][runtime] Support consuming multiple subpartitions on a single channel [flink]

2023-12-24 Thread via GitHub
TanYuxin-tyx commented on code in PR #23927: URL: https://github.com/apache/flink/pull/23927#discussion_r1435920016 ## flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/netty/NettyConnectionWriterImpl.java: ## @@ -30,24 +30,33 @@ public clas

Re: [PR] [FLINK-33879] Avoids the potential hang of Hybrid Shuffle during redistribution [flink]

2023-12-24 Thread via GitHub
TanYuxin-tyx commented on code in PR #23957: URL: https://github.com/apache/flink/pull/23957#discussion_r1435909606 ## flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java: ## @@ -244,6 +282,39 @@ pub

[jira] [Closed] (FLINK-33635) Some connectors can not compile in 1.19-SNAPSHOT

2023-12-24 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo closed FLINK-33635. -- Resolution: Fixed > Some connectors can not compile in 1.19-SNAPSHOT > ---

[jira] [Commented] (FLINK-33635) Some connectors can not compile in 1.19-SNAPSHOT

2023-12-24 Thread Weijie Guo (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17800229#comment-17800229 ] Weijie Guo commented on FLINK-33635: Yep, I have checked the nightly CI of ElasticSe

[jira] [Comment Edited] (FLINK-31830) Coalesce on nested fields with different nullabilities will get wrong plan

2023-12-24 Thread Jane Chan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-31830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17800227#comment-17800227 ] Jane Chan edited comment on FLINK-31830 at 12/25/23 2:06 AM: -

[jira] [Commented] (FLINK-31830) Coalesce on nested fields with different nullabilities will get wrong plan

2023-12-24 Thread Jane Chan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-31830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17800227#comment-17800227 ] Jane Chan commented on FLINK-31830: --- I've already responded above, please see the "Beh

Re: [PR] [FLINK-33930] Stateless stop job status exception [flink-kubernetes-operator]

2023-12-24 Thread via GitHub
hunter-cloud09 commented on PR #740: URL: https://github.com/apache/flink-kubernetes-operator/pull/740#issuecomment-1868659512 > Please add a proper PR description and a test. I am pretty sure that existing tests do not cover this as you changed the behaviour. Please see [issue](http

[jira] [Commented] (FLINK-33635) Some connectors can not compile in 1.19-SNAPSHOT

2023-12-24 Thread lincoln lee (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17800225#comment-17800225 ] lincoln lee commented on FLINK-33635: - [~Weijie Guo] Could you have a check on this?

Re: [PR] [doc-fix] - changed preference to settings for setting pycharm for pyflink ide setting section [flink]

2023-12-24 Thread via GitHub
adipolak commented on code in PR #23903: URL: https://github.com/apache/flink/pull/23903#discussion_r1435898859 ## docs/content/docs/flinkDev/ide_setup.md: ## @@ -260,7 +260,7 @@ versions. [Flake8](https://pypi.org/project/flake8/) is used to enforce some coding guidelines.

[jira] [Commented] (FLINK-31830) Coalesce on nested fields with different nullabilities will get wrong plan

2023-12-24 Thread Sergey Nuyanzin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-31830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17800216#comment-17800216 ] Sergey Nuyanzin commented on FLINK-31830: - [~qingyue] thanks for the response j

Re: [PR] [hotfix] [docs] Update README.md to remove Maven [flink]

2023-12-24 Thread via GitHub
snuyanzin commented on PR #23981: URL: https://github.com/apache/flink/pull/23981#issuecomment-1868594128 1. the PR is incomplete: removal requirement for maven is not enough since to make it reproducible there is no info how to enable it in IDE. In that case in terminal there will be wrapp

Re: [PR] [hotfix] [docs] Update README.md to remove Maven [flink]

2023-12-24 Thread via GitHub
asafm commented on PR #23981: URL: https://github.com/apache/flink/pull/23981#issuecomment-1868536450 @snuyanzin Why would you want users, to run with their installed Maven? If you want all users to have reproducible builds, they should only use the maven wrapper, pinned to a specific versi

[jira] (FLINK-33681) Display source/sink numRecordsIn/Out & numBytesIn/Out on UI

2023-12-24 Thread Zhanghao Chen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33681 ] Zhanghao Chen deleted comment on FLINK-33681: --- was (Author: zhanghao chen): After further investigation, it turned out to be a bug introduced by FLINK-30536, which changed the way to count

[jira] [Comment Edited] (FLINK-33681) Display source/sink numRecordsIn/Out & numBytesIn/Out on UI

2023-12-24 Thread Zhanghao Chen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17800174#comment-17800174 ] Zhanghao Chen edited comment on FLINK-33681 at 12/24/23 12:24 PM:

[jira] [Updated] (FLINK-33681) Display source/sink numRecordsIn/Out & numBytesIn/Out on UI

2023-12-24 Thread Zhanghao Chen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhanghao Chen updated FLINK-33681: -- Affects Version/s: 1.17.2 1.18.0 Issue Type: Bug (was: Impro

[jira] [Commented] (FLINK-33681) Display source/sink numRecordsIn/Out & numBytesIn/Out on UI

2023-12-24 Thread Zhanghao Chen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17800174#comment-17800174 ] Zhanghao Chen commented on FLINK-33681: --- After further investigation, it turned ou

Re: [PR] [FLINK-32073][checkpoint] Implement file merging in snapshot [flink]

2023-12-24 Thread via GitHub
curcur commented on PR #23514: URL: https://github.com/apache/flink/pull/23514#issuecomment-1868496305 CI seems Failed with "[99d7b97](https://github.com/apache/flink/pull/23514/commits/99d7b97cd5844ec6362b30d8f9134509b72cb647)" -- This is an automated message from the Apache Git Service.

[jira] [Updated] (FLINK-33450) Implement JdbcAutoScalerStateStore

2023-12-24 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-33450: --- Labels: pull-request-available (was: ) > Implement JdbcAutoScalerStateStore > -

[PR] [FLINK-33450][autoscaler] Support the JDBCAutoScalerStateStore [flink-kubernetes-operator]

2023-12-24 Thread via GitHub
1996fanrui opened a new pull request, #741: URL: https://github.com/apache/flink-kubernetes-operator/pull/741 It's still developing, this is a draft PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to