Sounds like the parallel consumer is what you want:
https://github.com/confluentinc/parallel-consumer
Or at least decide why that won't work before writing something yourself.
- alex
On Tue, Oct 3, 2023 at 4:20 PM Sree Sanjeev Kandasamy Gokulrajan <
sreesanjee...@gmail.com> wrote:
> Hey Neeraj
Hey Neeraj
I had a kafka assignment where I had to consume from a single partition.
When I was looking to scale the application, I was planning to consume the
partition with more than one consumer when I came across this issue.
Regards,
Sree Sanjeev.
On Tue, Oct 3, 2023 at 3:48 PM Neeraj Vaidya
Hi Sree Sanjeev,
Other than the info from Colt, why can't you just increase partitions and scale
the number of consumers accordingly ?
Or build your application as a KStreams application and take advantage of
horizontal and vertical scalability ?
Regards,
Neeraj
On Wednesday, 4 October,
Hi,
Thank you all for participating in the 3.6.0 release candidate vote threads.
This vote passes with 11 +1 votes(4 binding) and no 0 or -1 votes.
+1 votes
PMC Members:
* Luke Chen
* Bill Bejeck
* Chris Egerton
* Justine Olshan
Community:
* Jakub Scholz
* Federico Valeri
* Divij Vaidya
* Kamal
Hi Team,
In one of our environment during helm upgrade of our charts which includes
Strimzi Kafka chart, we see zookeeper pod 1 went for Crashloopbackoff state.
zookeeper-0 and zookeeper-2 are running fine. We never seen this behaviour till
date.
srini-kf-op-sz-zookeeper-1
Hi Sree.
Yeah, parallel-processing-per-partition requirement often arises
particularly when the throughput is limited due to external I/O latency and
there are some solutions:
- https://github.com/line/decaton
* provides per-key ordering guarantee (or can be unordered if no
ordering is necess
Hello,
Currently, you cannot have more than one consumer consuming from a
partition in Kafka. However, there is active discussion about a proposal to
enable queueing semantics for Kafka, which is a superset of the feature you
are talking about. Feel free to check KIP-932:
https://cwiki.apache.org/
I'm interested in knowing if we can achieve parallelism by allowing
multiple consumers to subscribe to a single partition of a topic.
To explore potential solutions, I'm considering the following approach.
1. Implementing a locking mechanism to control access to the offsets in a
partition to avoi
Thanks Divij for the observations.
KAFKA-15483 is added to address the missing metrics related to KRaft
and ZK to KRaft migration. Please leave a comment at
https://github.com/apache/kafka-site/pull/548 in which related metrics
documentation is added.
https://issues.apache.org/jira/browse/KAFKA-1
Hello All,
We are facing one issue in the 3.4.1 kafka version cluster. We have
implemented 3 brokers cluster on a single node server in the kubernetes
environment. Whenever one of the broker goes offline then we are facing
below issue. So, please help us to resolve these issue. Thanks to all in
ad
10 matches
Mail list logo
