Re: Regarding an issue with message distributions in kafka.

Hi Puneet, Thanks for the question. It sounds like either the three consumers have different group.id configurations or they are using “assign” instead of “subscribe”. You are correct, if the consumers are all in the same consumer group and subscribed to the same topic, Kafka will only assign

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1374

See Changes: -- [...truncated 513133 lines...] [2022-11-23T16:43:32.683Z] [2022-11-23T16:43:32.683Z] Gradle Test Run :streams:integrationTest > Gradle Test Executor 165 >

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

Hi Alex, Thanks for the feedback. I've updated the discussion of OOM issues by describing how we'll handle it. Here's the new text: To mitigate this, we will automatically force a Task commit if the total > uncommitted records returned by > StateStore#approximateNumUncommittedEntries() exceeds

Re: Regarding an issue with message distributions in kafka.

Hello Puneet, Why do you need multiple consumers from the same group consuming from the same partition? I am trying to think of a compiling reason for such a use case... It makes more sense to me to have a thread pool to process the records fetched from a single consumer. This is a great article

Regarding an issue with message distributions in kafka.

Hi. In our app we have integrated kafka to publish and subscribe to the events. Based on the documentation and distribution policy we designed our application where we are facing some contradictory behavior. As per the documentation at a given time there can be a maximum of one consumer per partit

Re: [DISCUSS] KIP-889 Versioned State Stores

Hi all, Thanks for the KIP, Victoria! I have a couple of comments. 1. delete(key) I think delete(key) should not remove all versions of a key. We should use it to close the validity interval of the last version. Assuming we have records of different versions for key A: (A, e, 0, 2), (A, f, 2,

[jira] [Resolved] (KAFKA-12610) Resources on classpath break PluginClassLoader isolation

[ https://issues.apache.org/jira/browse/KAFKA-12610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-12610. --- Fix Version/s: 3.4.0 Resolution: Fixed > Resources on classpath break PluginClassLoad

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

Hey Nick, Thank you for the KIP! With such a significant performance degradation in the secondary store approach, we should definitely consider WriteBatchWithIndex. I also like encapsulating checkpointing inside the default state store implementation to improve performance. +1 to John's comment t

Re: [DISCUSS] KIP-881: Rack-aware Partition Assignment for Kafka Consumers

Hi Artem, Thanks for reviewing the KIP. The client doesn't rely on a particular replica assignment logic in the broker. Instead, it matches the actual rack assignment for partitions from the metadata with consumer racks. So there is no assumption in the client implementation about the use of RackA

Re: [DISCUSS] KIP-889 Versioned State Stores

Hi Vicky, Thanks for your response! I would just use numbers to refer to your comments. 1) Thanks for your response. Even I am not totally sure whether these should be supported via IQv2 or via store interface. That said, I wouldn't definitely qualify this as blocking the KIP for sure so we can