[jira] [Resolved] (KAFKA-18058) Integrate offset cleanup algorithm with share coordinator

[ https://issues.apache.org/jira/browse/KAFKA-18058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-18058. -- Fix Version/s: 4.0.0 Resolution: Fixed > Integrate offset cleanup algorithm wit

[jira] [Created] (KAFKA-18218) Trogdor system test can't find coordinator

TaiJuWu created KAFKA-18218: --- Summary: Trogdor system test can't find coordinator Key: KAFKA-18218 URL: https://issues.apache.org/jira/browse/KAFKA-18218 Project: Kafka Issue Type: Test

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

Hi All, The following changes are made to the KIP while implementing the changes to Kafka Streams. 1. We have changed AutoOffsetReset.duration(...) API to AutoOffsetReset.byDuration(...) 2. Added "none" reset strategy as an option for the newly added org.apache.kafka.streams.AutoOffsetReset class

[jira] [Created] (KAFKA-18217) Slow HWM/LSO update might have subtle effect on the consumer lag reporting

Philip Nee created KAFKA-18217: -- Summary: Slow HWM/LSO update might have subtle effect on the consumer lag reporting Key: KAFKA-18217 URL: https://issues.apache.org/jira/browse/KAFKA-18217 Project: Kafka

[jira] [Created] (KAFKA-18216) High water mark or last stable offset aren't always updated after a fetch request is completed

Philip Nee created KAFKA-18216: -- Summary: High water mark or last stable offset aren't always updated after a fetch request is completed Key: KAFKA-18216 URL: https://issues.apache.org/jira/browse/KAFKA-18216

[jira] [Resolved] (KAFKA-12825) Remove Deprecated method StreamsBuilder#addGlobalStore

[ https://issues.apache.org/jira/browse/KAFKA-12825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-12825. - Resolution: Duplicate > Remove Deprecated method StreamsBuilder#addGlobalStore > ---

[jira] [Reopened] (KAFKA-12825) Remove Deprecated method StreamsBuilder#addGlobalStore

[ https://issues.apache.org/jira/browse/KAFKA-12825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-12825: - > Remove Deprecated method StreamsBuilder#addGlobalStore > -

[jira] [Resolved] (KAFKA-12831) Remove Deprecated method StateStore#init

[ https://issues.apache.org/jira/browse/KAFKA-12831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-12831. - Resolution: Duplicate > Remove Deprecated method StateStore#init > -

[jira] [Reopened] (KAFKA-12831) Remove Deprecated method StateStore#init

[ https://issues.apache.org/jira/browse/KAFKA-12831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-12831: - > Remove Deprecated method StateStore#init > > >

Jenkins build became unstable: Kafka » Kafka PowerPC Daily » test-powerpc #145

See

[jira] [Resolved] (KAFKA-18186) add sourceCompatibility back to build.gradle to allow idea to configure suitable language level automatically

[ https://issues.apache.org/jira/browse/KAFKA-18186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-18186. Fix Version/s: 4.0.0 Resolution: Fixed > add sourceCompatibility back to build.grad

[jira] [Resolved] (KAFKA-18215) Accept plugin.version configurations for connector and converters

[ https://issues.apache.org/jira/browse/KAFKA-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-18215. - Fix Version/s: 4.0.0 Assignee: Snehashis Pal Resolution: Fixed > Accept plugin.v

[jira] [Created] (KAFKA-18215) Accept plugin.version configurations for connector and converters

Greg Harris created KAFKA-18215: --- Summary: Accept plugin.version configurations for connector and converters Key: KAFKA-18215 URL: https://issues.apache.org/jira/browse/KAFKA-18215 Project: Kafka

[jira] [Resolved] (KAFKA-18015) Add duration based offset reset option for Kafka Streams

[ https://issues.apache.org/jira/browse/KAFKA-18015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-18015. - Resolution: Fixed > Add duration based offset reset option for Kafka Streams > -

Re: [jira] [Created] (KAFKA-18204) Upgrade to rocksdb 8.x+ (ideally 9.x)

Swikar, thanks for reaching out. Yes, we want to address this for the upcoming 4.0 release. some interesting things that might be relevant to this upgrade. Can you elaborate? Curious to learn more. Otherwise, let's move the discussion on the Jira ticket. -Matthias > whether it's researc

Re: [VOTE] KIP-1111: Enforcing Explicit Naming for Kafka Streams Internal Topics

+1 (binding) thanks Sebastien! -Sophie On Tue, Dec 10, 2024 at 9:26 AM Bill Bejeck wrote: > Hi Sébastien, > > Thanks for the KIP! > > +1(binding) > > Regards, > Bill > > On Fri, Dec 6, 2024 at 4:33 AM Bruno Cadonna wrote: > > > Hi Sébastien, > > > > +1 (binding) > > > > Many Thanks! > > Bruno

Re: [DISCUSS] KIP-1111: Enforcing Explicit Naming for Kafka Streams Internal Topics

Few final thoughts: 1. I agree with Matthias, this config merits HIGH importance 2. Totally makes sense to leave the DSL/StreamsBuilder config stuff for a separate KIP. Let me know if you want to chat about that idea more if/when you want to pick that up Thanks for the KIP! On Tue, Dec 10, 2024

[jira] [Resolved] (KAFKA-13713) Tech Debt: keep StreamThread and TopologyMetadata's view of the topology in sync

[ https://issues.apache.org/jira/browse/KAFKA-13713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-13713. Resolution: Won't Fix > Tech Debt: keep StreamThread and TopologyMetadata'

[jira] [Resolved] (KAFKA-13282) Draft final NamedTopology API and publish a KIP

[ https://issues.apache.org/jira/browse/KAFKA-13282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-13282. Resolution: Won't Fix > Draft final NamedTopology API and publish a KIP >

[jira] [Resolved] (KAFKA-13645) Support the TopologyTestDriver with modular topologies

[ https://issues.apache.org/jira/browse/KAFKA-13645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-13645. Resolution: Won't Fix > Support the TopologyTestDriver with modular topolo

[jira] [Resolved] (KAFKA-13644) Support global state stores with modular topologies

[ https://issues.apache.org/jira/browse/KAFKA-13644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-13644. Resolution: Won't Fix > Support global state stores with modular topologie

[jira] [Resolved] (KAFKA-13283) Migrate experimental feature to public API

[ https://issues.apache.org/jira/browse/KAFKA-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-13283. Resolution: Won't Fix > Migrate experimental feature to public API > -

[jira] [Resolved] (KAFKA-13643) Replace "NamedTopology" with "ModularTopology" in the codebase

[ https://issues.apache.org/jira/browse/KAFKA-13643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-13643. Resolution: Won't Fix > Replace "NamedTopology" with "ModularTopology" in

[jira] [Resolved] (KAFKA-13712) Make topology addition/removal atomic so we can roll back if request fails

[ https://issues.apache.org/jira/browse/KAFKA-13712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-13712. Resolution: Won't Fix > Make topology addition/removal atomic so we can ro

[jira] [Resolved] (KAFKA-12648) Experiment with resilient isomorphic topologies

[ https://issues.apache.org/jira/browse/KAFKA-12648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-12648. Resolution: Won't Fix Deprecated named topologies in 4.0 with the intentio

[jira] [Resolved] (KAFKA-13711) Fix bugs with input topic management to support pattern subscription fully

[ https://issues.apache.org/jira/browse/KAFKA-13711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-13711. Resolution: Won't Fix > Fix bugs with input topic management to support pa

[jira] [Resolved] (KAFKA-18021) Disabled MirrorCheckpointConnector throws RetriableException on task config generation

[ https://issues.apache.org/jira/browse/KAFKA-18021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-18021. - Fix Version/s: 4.0.0 Resolution: Fixed > Disabled MirrorCheckpointConnector throws Retria

Re: [DISCUSS] KIP-1114 Introducing Chunk in Partition

Hey there, Thanks for sharing this KIP. I took a brief look and had a similar thought as David about this with wondering how this compares to using tiered storage. I was also thinking about fetching from any replica (KIP-392) related to this as well. I think overall, Chunk could be useful, but it

[jira] [Resolved] (KAFKA-15387) Deprecate and remove Connect's redundant task configurations retrieval endpoint

[ https://issues.apache.org/jira/browse/KAFKA-15387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-15387. - Resolution: Fixed > Deprecate and remove Connect's redundant task configurations retrieval > en

[jira] [Resolved] (KAFKA-17040) Unknown telemetry state: TERMINATED thrown when closing AsyncKafkaConsumer

[ https://issues.apache.org/jira/browse/KAFKA-17040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans resolved KAFKA-17040. Resolution: Fixed > Unknown telemetry state: TERMINATED thrown when closing AsyncKafkaCons

Re: [DISCUSS] Apache Kafka 4.0.0 release

> We ran onto issues with the Connect system tests caused by AsyncConsumer (https://issues.apache.org/jira/browse/KAFKA-18160). I believe the patch for *KIP-653* is almost complete. However, it would be ideal to pass all end-to-end (e2e) tests, as it introduces significant changes. Except for *KA

[jira] [Resolved] (KAFKA-16618) Update the RPC for ConsumerGroupHeartbeatRequest and ConsumerGroupHeartbeatResponse

[ https://issues.apache.org/jira/browse/KAFKA-16618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-16618. - Assignee: (was: Colin McCabe) Resolution: Duplicate > Update the RPC for ConsumerGroup

Re: [DISCUSS] Apache Kafka 4.0.0 release

Hi David, There's also KIP-653: Upgrade log4j to log4j2 which we have not been able to merge yet. We ran onto issues with the Connect system tests caused by AsyncConsumer (https://issues.apache.org/jira/browse/KAFKA-18160). Thanks, Mickael On Wed, Dec 11, 2024 at 7:41 PM José Armando García Sanc

Re: [DISCUSS] KIP-1114 Introducing Chunk in Partition

Hi David: Thanks for the helping info. Appreciated. I believe the chunk is still needed in spite of tiered storage as this makes Kafka more 'complete'. Let me extend the motivation section. I wasn't aware we are busy on Kafka 4.0. If that's the case I will see if can contribute on that while wai

Re: [DISCUSS] Apache Kafka 4.0.0 release

Hi David, Happy feature freeze day! See my comments below. On Wed, Dec 11, 2024 at 3:46 AM David Jacot wrote: > Thanks for reaching out to me about KIP-966. It seems to me that > https://github.com/ahuang98/kafka/pull/2 is a pretty large change for after > the feature freeze. How do you quantify

[jira] [Created] (KAFKA-18214) TestUtils#waitForCondition does not honor the maxWaitMs

Chia-Ping Tsai created KAFKA-18214: -- Summary: TestUtils#waitForCondition does not honor the maxWaitMs Key: KAFKA-18214 URL: https://issues.apache.org/jira/browse/KAFKA-18214 Project: Kafka I

[jira] [Created] (KAFKA-18213) Convert CustomQuotaCallbackTest to KRaft

Mickael Maison created KAFKA-18213: -- Summary: Convert CustomQuotaCallbackTest to KRaft Key: KAFKA-18213 URL: https://issues.apache.org/jira/browse/KAFKA-18213 Project: Kafka Issue Type: Sub-

Re: [PR] MINOR: add jeffkbkim to committers [kafka-site]

chia7712 merged PR #653: URL: https://github.com/apache/kafka-site/pull/653 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@kafka.apache

[PR] MINOR: add jeffkbkim to committers [kafka-site]

jeffkbkim opened a new pull request, #653: URL: https://github.com/apache/kafka-site/pull/653  -- This is an automated message from the Apache Git Service. To respond to the message, p

[jira] [Resolved] (KAFKA-16437) Upgrade to Jakarta and JavaEE 10 in Kafka 4.0 (KIP-1032)

[ https://issues.apache.org/jira/browse/KAFKA-16437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16437. Resolution: Fixed > Upgrade to Jakarta and JavaEE 10 in Kafka 4.0 (KIP-1032) > ---

Re: [DISCUSS] KIP-1034: Dead letter queue in Kafka Streams

Hi, BC6 I agree that there is no risk with this parameter. MEDIUM level is a good choice. BC7 response is indeed less confusing and easier to "read" BC8 For the same reason as BC7, it might be a good idea. Since you marked your comment as nit-picking, I will wait for other feedback before updati

Re: [DISCUSS] KIP-1097: error record reporter

Bumping the thread On Sun, Dec 1, 2024 at 8:33 AM Anton Liauchuk wrote: > Hi > Thank you for your feedback. > I have numbered the questions to simplify communication. > > 1. What sort of implementation do you have in mind for this interface? >> What use-case does this interface enable that is no

[jira] [Created] (KAFKA-18212) Rolling-upgrade broker causes unnecessary re-balance on classic consumer

Chia-Ping Tsai created KAFKA-18212: -- Summary: Rolling-upgrade broker causes unnecessary re-balance on classic consumer Key: KAFKA-18212 URL: https://issues.apache.org/jira/browse/KAFKA-18212 Project:

[jira] [Created] (KAFKA-18210) Convert ConsumerRebootstrapTest to KRaft

Mickael Maison created KAFKA-18210: -- Summary: Convert ConsumerRebootstrapTest to KRaft Key: KAFKA-18210 URL: https://issues.apache.org/jira/browse/KAFKA-18210 Project: Kafka Issue Type: Sub-

[jira] [Created] (KAFKA-18211) ClassGraph scanning does not correctly find isolated connect plugins

Snehashis Pal created KAFKA-18211: - Summary: ClassGraph scanning does not correctly find isolated connect plugins Key: KAFKA-18211 URL: https://issues.apache.org/jira/browse/KAFKA-18211 Project: Kafka

[jira] [Created] (KAFKA-18209) Cleanup __transaction_state config logic

Mickael Maison created KAFKA-18209: -- Summary: Cleanup __transaction_state config logic Key: KAFKA-18209 URL: https://issues.apache.org/jira/browse/KAFKA-18209 Project: Kafka Issue Type: Sub-

Re: [DISCUSS] KIP-1034: Dead letter queue in Kafka Streams

Hi, Another suggestion came to my mind. BC9 You could use "resume" instead of "continue". Thus, you would not need to use continueProcessing(), failProcessing(), and retryProcessing() but simply resume(), fail(), retry(). Regarding the enum value CONTINUE, we do not need to change it to RESUME

[jira] [Created] (KAFKA-18208) Refactor TransactionLog logic

Mickael Maison created KAFKA-18208: -- Summary: Refactor TransactionLog logic Key: KAFKA-18208 URL: https://issues.apache.org/jira/browse/KAFKA-18208 Project: Kafka Issue Type: Task

Re: [VOTE] KIP-1034: Dead letter queue in Kafka Streams

Hi all, I replied to the DISCUSS thread since I believe that we should keep this thread for recasting our votes after the discussion. Best, Bruno On 09.12.24 09:55, Sebastien Viale wrote: Hello, I have some concerns about the naming: 1. continue() is not feasible because continue is a re

Re: [DISCUSS] KIP-1034: Dead letter queue in Kafka Streams

Hi Sébastien, Thanks for the updates! BC6 Why is the importance of ERRORS_DEADLETTERQUEUE_TOPIC_NAME_CONFIG HIGH? Is it really something a user should change before going to production? The only risk that I see is that they will not use a dead letter queue which I think does not put the app or

[jira] [Resolved] (KAFKA-18179) Move AsyncOffsetReadFutureHolder to storage module

[ https://issues.apache.org/jira/browse/KAFKA-18179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-18179. Fix Version/s: 4.0.0 Resolution: Fixed > Move AsyncOffsetReadFutureHolder to storag

[jira] [Created] (KAFKA-18207) Create TransactionCoordinatorRecordSerde for handling transaction records

Mickael Maison created KAFKA-18207: -- Summary: Create TransactionCoordinatorRecordSerde for handling transaction records Key: KAFKA-18207 URL: https://issues.apache.org/jira/browse/KAFKA-18207 Project

[jira] [Resolved] (KAFKA-14410) Allow connect runtime to run multiple versions of a connector.

[ https://issues.apache.org/jira/browse/KAFKA-14410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Snehashis Pal resolved KAFKA-14410. --- Resolution: Duplicate > Allow connect runtime to run multiple versions of a connector. > --

[jira] [Created] (KAFKA-18206) EmbeddedKafkaCluster must set features

David Jacot created KAFKA-18206: --- Summary: EmbeddedKafkaCluster must set features Key: KAFKA-18206 URL: https://issues.apache.org/jira/browse/KAFKA-18206 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-1114 Introducing Chunk in Partition

Hi, Thanks for the KIP. The community is pretty busy with the Apache Kafka 4.0 release so I suppose that no one really had the time to engage in reviewing the KIP yet. Sorry for this! I just read the motivation section. I think that it is an interesting idea. However, I wonder if this is still ne

Re: [DISCUSS] Apache Kafka 4.0.0 release

Hi folks, Happy feature freeze day! Please remember that the big chunks of your KIPs must be merged by today. In practice, I plan to cut the release branch on Monday so you have a few more days. If you cannot make it, please reachout to me to discuss how to proceed. I will go and remove the KIPs w

[jira] [Resolved] (KAFKA-18204) Upgrade to rocksdb 8.x+ (ideally 9.x)

[ https://issues.apache.org/jira/browse/KAFKA-18204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-18204. --- Fix Version/s: 4.0.0 Resolution: Duplicate > Upgrade to rocksdb 8.x+ (ideally 9.x) >

[jira] [Created] (KAFKA-18205) Refactor share coordinator and related classes for better readability and organization

Sushant Mahajan created KAFKA-18205: --- Summary: Refactor share coordinator and related classes for better readability and organization Key: KAFKA-18205 URL: https://issues.apache.org/jira/browse/KAFKA-18205

[jira] [Resolved] (KAFKA-18057) Algorithm to calculate redundant interval of offsets for cleanup

[ https://issues.apache.org/jira/browse/KAFKA-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushant Mahajan resolved KAFKA-18057. - Resolution: Won't Fix This is addressed in a single PR for issue KAFKA-18058 > Algorith