Detail I forgot to mention is that I am using EOS, so the streams application
was consistently getting this error causing it to restart that task, which
obviously would bottle neck everything.
Once I increased the threads on the brokers the epoch error subsided until the
volume increased.
Right
I'm not one of the real experts on the Producer and even further from one
with broker performance, so someone else may need to chime in for that, but
I did have a few questions:
What specifically are you unsatisfied with w.r.t the performance? Are you
hoping for a higher throughput
of your Streams
Hi,
I have a kafka streams application deployed on 5 nodes and with full traffic I
am getting the error message:
org.apache.kafka.common.errors.InvalidProducerEpochException: Producer
attempted to produce with an old epoch.
Written offsets would not be recorded and no more records would be sent
Hi All,
We are using AWS MSK with kafka version 2.6.1. There is a compacted topic
with the below configurations. After reading the documentation my
understanding was that null values in the topic can be removed using delete
retention time but I can see months old keys having null values. Is there
Hi All,
Has anyone implemented rack awareness with AWS MSK? In the broker
configuration I see the following values (use1-az2, use1-az4 & use1-az6).
How are these values determined and how can we inject these values to java
consumers? Thanks!
Regards,
Navneeth
I don't think the Confluent self-balancing feature works if you have your
broker data in multiple directories anyway - it's expecting a single dir
per broker and will try and keep the data balanced between brokers. Also
just as an aside, I'm not sure there's much value in using multiple
directorie
There's Cruise Control, https://github.com/linkedin/cruise-control, which
is open-source and could help with automated balancing.
On Thu, Oct 27, 2022 at 10:26 AM wrote:
> Auto rebalancing is a very important feature to run Kafka in a production
> environment. Given the confluent already have th
Auto rebalancing is a very important feature to run Kafka in a production
environment. Given the confluent already have this feature, are there any space
that the open source version could have this feature as well?
Or, is it the idea that opensource version shouldn't be used in a high load
prod
