Re: Error in kafka streams: The producer attempted to use a producer id which is not currently assigned to its transactional id

Matthias, Is there any work around after the stream goes into error because of above issue like attaching a StateListener on the StreamBuilder and restart the stream in case of ERROR state? Right now, we need to start the pod that hosts the application which won't be feasible when the application

Re: Error in kafka streams: The producer attempted to use a producer id which is not currently assigned to its transactional id

Well, it's for sure EOS related, but it seems to be a different root cause. I am not aware of any related bug. -Matthias On 9/9/20 4:29 AM, Pushkar Deole wrote: > Hi Matthias, > > We are using confluent kafka and upgraded to confluent version 5.5.0 which > I believe maps to 2.5.0 of apache kaf

Re: Scaling issues with MSK on AWS

Hi Arti, >From your description, it sounds like your per-partition throughput is extremely high (380 MB/s across three partitions is ~125 MB/s per partition). A more typical high-end range would be 5-10 MB/s per partition. Each follower gets mapped to a single replica fetcher thread on the followe

Scaling issues with MSK on AWS

Hi, We have been evaluating Managed Streaming for Kafka (MSK) on AWS for a use-case that requires high-speed data ingestion of the order of millions of messages (each ~1 KB size) per second. We ran into some issues when testing this case. Context: To start with, we have set up single topic wit

Apache Kafka website "videos" page clarification

Hi all! I have just noticed the new content on the Apache Kafka website about books, papers, podcasts, and videos ... congratulations and great works to put them all together!! It's an impressive list!! On the videos page I read this: The following talks, with video recordings and slides availab

InvalidRecordException: Incorrect declared batch size, records still remaining in file

Hello list, we encountered the error in subject yesterday for the first time and are looking for an explanation. We can't find any hints of what might have happened. The broker logs contain nothing unusual and all other metrics (hardware and software) were "as always" at the time of the incident.

Re: Error in kafka streams: The producer attempted to use a producer id which is not currently assigned to its transactional id

Hi Matthias, We are using confluent kafka and upgraded to confluent version 5.5.0 which I believe maps to 2.5.0 of apache kafka. We tested on few labs by keeping the solution idle for few days and didn't observe the issue. However on one of the labs we observed issue again recently, this is the e