Re: [ANNOUNCE] Apache Kafka 3.9.1

2025-05-21 Thread Dimitry Lvovsky
On Wed, May 21, 2025 at 5:37 AM TengYao Chi wrote: > The Apache Kafka community is pleased to announce the release for > Apache Kafka 3.9.1 > > This bug-fix release has several fixes, and most importantly, it adds > Java 23 support. > > All of the changes in this release can be found in the relea

Re: org.apache.kafka.common.KafkaException: org.apache.kafka.common.serialization.ByteArraySerializer is not an instance of org.apache.kafka.common.serialization.Serializer

2020-01-27 Thread Dimitry Lvovsky
unsubscribe On Mon, Jan 27, 2020 at 8:51 AM wrote: > > > Hi, I would appreciate any help on this. > > Thanks a stack! > > Buks > > > > > org.apache.kafka > kafka-clients > ${kafka.version} > > > > > 2.3.1 > > > > > org.springframework.boot > spring-boot-starter-parent >

Re: Failed to rebalance -- Unexpected error in InitProducerIdResponse

2019-09-03 Thread Dimitry Lvovsky
be other apps are experiencing diff issues around the same time. > Without more info, if difficult to triage. > > Hope that helps, > -- > Jonathan > > > > > On Tue, Sep 3, 2019 at 12:46 PM Dimitry Lvovsky > wrote: > > > Thanks, Jonathan for the quick response.

Re: Failed to rebalance -- Unexpected error in InitProducerIdResponse

2019-09-03 Thread Dimitry Lvovsky
> > Cheers! > > > > On Tue, Sep 3, 2019 at 12:38 PM Dimitry Lvovsky > wrote: > > > Hi all, > > We have a streaming app running for over a year now without much incident > > until recently. Suddenly the process keeps failing after a minute or so > > with

Failed to rebalance -- Unexpected error in InitProducerIdResponse

2019-09-03 Thread Dimitry Lvovsky
Hi all, We have a streaming app running for over a year now without much incident until recently. Suddenly the process keeps failing after a minute or so with a failed to rebalance exception caused by Unexpected error in InitProducerIdResponse; The server experienced an unexpected error when pro

Re: Too many commits

2019-04-25 Thread Dimitry Lvovsky
;m sorry I don't have the link to it off hand. Hope this helps. On Thu, Apr 25, 2019 at 9:26 AM yuvraj singh <19yuvrajsing...@gmail.com> wrote: > Yeah it's taking time , that's why I am doing manually commits to achieve > at least once semantics . > > >

Re: Too many commits

2019-04-25 Thread Dimitry Lvovsky
Are your processes taking a long time between commits — does consuming each message take a long while? On Thu, Apr 25, 2019 at 08:50 yuvraj singh <19yuvrajsing...@gmail.com> wrote: > Hi all , > > In my application i am committing every offset to kafka one by one and my > max poll size is 30 . I a

Re: How to be notified by Kafka stream during partitions rebalancing

2019-04-04 Thread Dimitry Lvovsky
You can detect state changes in your streaming app by implementing KafkaStreams.StateListener, and then registering that with your KafkaStreeams Object e.g new KafkaStreams(...).setStateListener(); Hope this helps. On Thu, Apr 4, 2019 at 10:52 AM Pierre Coquentin wrote: > Hi, > > We have a cach

Re: Something like a unique key to prevent same record from being inserted twice?

2019-04-03 Thread Dimitry Lvovsky
I've done this using kafka streams: specifically, I created a processor, and use a keystore (a functionality of streams) to save/check for keys and only forwarding messages that were not in the keystore. Since the keystore is in memory, and backed by the local filesystem on the node the processor i

Re: Need help to find references to antipatterns/pitfalls/incorrect ways to use Kafka

2019-04-01 Thread Dimitry Lvovsky
Going off of what Hans mentioned, I don't see any reason for 200,000 partitions...you don't need one partition per POS. You can have all of your pos listening to one partition and each pos agent having a unique group id. The POS agent only processes the messages that are relevant to him, and simpl

Re: Dynamic Partitioning

2019-01-09 Thread Dimitry Lvovsky
One approach you can take is to set an upper bound number of partitions a priori. Imagine your key was the username. If you had 2 partitions in your topic and 4 users writing messages, then Kafka would split these messages between two partitions of the topic (assuming the usernames are unique). Fo