[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17860921#comment-17860921 ] Hilmi Al Fatih commented on FLINK-34554: [~fpaul] From theoretical perspective,

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17827657#comment-17827657 ] Hilmi Al Fatih commented on FLINK-34554: Hi [~martijnvisser], I just reproduced

[jira] [Updated] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hilmi Al Fatih updated FLINK-34554: --- Attachment: image-2024-03-16-17-17-16-152.png > Using EXACTLY_ONCE with KafkaSink cause brok

[jira] [Comment Edited] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824769#comment-17824769 ] Hilmi Al Fatih edited comment on FLINK-34554 at 3/8/24 12:21 PM: -

[jira] [Comment Edited] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824769#comment-17824769 ] Hilmi Al Fatih edited comment on FLINK-34554 at 3/8/24 12:19 PM: -

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824769#comment-17824769 ] Hilmi Al Fatih commented on FLINK-34554: I see. Let me get back to you by reprod

[jira] [Comment Edited] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824763#comment-17824763 ] Hilmi Al Fatih edited comment on FLINK-34554 at 3/8/24 11:58 AM: -

[jira] [Comment Edited] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824763#comment-17824763 ] Hilmi Al Fatih edited comment on FLINK-34554 at 3/8/24 11:58 AM: -

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824763#comment-17824763 ] Hilmi Al Fatih commented on FLINK-34554: Honestly I am afraid if you will remove

[jira] [Comment Edited] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824757#comment-17824757 ] Hilmi Al Fatih edited comment on FLINK-34554 at 3/8/24 11:49 AM: -

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824757#comment-17824757 ] Hilmi Al Fatih commented on FLINK-34554: {quote}You're not using the latest vers

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824749#comment-17824749 ] Hilmi Al Fatih commented on FLINK-34554: > The only thing that you could try is

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824748#comment-17824748 ] Hilmi Al Fatih commented on FLINK-34554: [~martijnvisser] IIUC my situation corr

[jira] [Commented] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824737#comment-17824737 ] Hilmi Al Fatih commented on FLINK-34554: Hi [~martijnvisser] , yes. We stop with

[jira] [Updated] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hilmi Al Fatih updated FLINK-34554: --- Description: Flink version: 1.17.1 Kafka Broker version: 2.7.1 * 4 GB heap memory for each

[jira] [Updated] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hilmi Al Fatih updated FLINK-34554: --- Description: Flink version: 1.17.1 Kafka Broker version: 2.7.1 * 4 GB heap memory for each

[jira] [Updated] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hilmi Al Fatih updated FLINK-34554: --- Priority: Blocker (was: Minor) > Using EXACTLY_ONCE with KafkaSink cause broker's OOM due t

[jira] [Updated] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

[ https://issues.apache.org/jira/browse/FLINK-34554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hilmi Al Fatih updated FLINK-34554: --- Issue Type: Improvement (was: New Feature) > Using EXACTLY_ONCE with KafkaSink cause broker

[jira] [Created] (FLINK-34554) Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint

Hilmi Al Fatih created FLINK-34554: -- Summary: Using EXACTLY_ONCE with KafkaSink cause broker's OOM due to newly created transactionalId per checkpoint Key: FLINK-34554 URL: https://issues.apache.org/jira/browse/F

[jira] [Commented] (FLINK-31483) Implement Split Deletion Support in Flink Kafka Connector

[ https://issues.apache.org/jira/browse/FLINK-31483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17785733#comment-17785733 ] Hilmi Al Fatih commented on FLINK-31483: Hi [~ruibin] [~ruanhang1993] Any update

[jira] [Commented] (FLINK-30490) Deleted topic from KafkaSource is still included in subsequent restart from savepoint

[ https://issues.apache.org/jira/browse/FLINK-30490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700951#comment-17700951 ] Hilmi Al Fatih commented on FLINK-30490: Hi [~ruibin]. Thanks for reply. I wonde

[jira] [Updated] (FLINK-30490) Deleted topic from KafkaSource is still included in subsequent restart from savepoint

[ https://issues.apache.org/jira/browse/FLINK-30490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hilmi Al Fatih updated FLINK-30490: --- Description: It seems that KafkaSource does not handle the removed topic partitions after r

[jira] [Updated] (FLINK-30490) Deleted topic from KafkaSource is still included in subsequent restart from savepoint

[ https://issues.apache.org/jira/browse/FLINK-30490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hilmi Al Fatih updated FLINK-30490: --- Description: It seems that KafkaSource does not handle the removed topic partitions after r

[jira] [Created] (FLINK-30490) Deleted topic from KafkaSource is still included in subsequent restart from savepoint

Hilmi Al Fatih created FLINK-30490: -- Summary: Deleted topic from KafkaSource is still included in subsequent restart from savepoint Key: FLINK-30490 URL: https://issues.apache.org/jira/browse/FLINK-30490

[jira] [Updated] (FLINK-29270) Wrong metrics kafka producer (FlinkKafkaProducer, KafkaSink) in EXACTLY_ONCE

[ https://issues.apache.org/jira/browse/FLINK-29270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Muhammad Hilmi Al Fatih updated FLINK-29270: Description: I did a very simple kafka-to-kafka pipeline with flink 1.14.4

[jira] [Updated] (FLINK-29270) Wrong metrics kafka producer (FlinkKafkaProducer, KafkaSink) in EXACTLY_ONCE

[ https://issues.apache.org/jira/browse/FLINK-29270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Muhammad Hilmi Al Fatih updated FLINK-29270: Description: I did a very simple kafka-to-kafka pipeline with flink 1.14.4

[jira] [Created] (FLINK-29270) Wrong metrics kafka producer (FlinkKafkaProducer, KafkaSink) in EXACTLY_ONCE

Muhammad Hilmi Al Fatih created FLINK-29270: --- Summary: Wrong metrics kafka producer (FlinkKafkaProducer, KafkaSink) in EXACTLY_ONCE Key: FLINK-29270 URL: https://issues.apache.org/jira/browse/FLINK-29270