Kafka can only do so much. Kafka’s High level consumer API does guarantee
delivery at least once to a living consumer’s message consumption
function. Kafka can't guarantee that the business logic that handles that
message won’t hang or do things to circumvent it’s guarantees.
But, since we are t
This is not good solution to monitor and kill the bad consumer, if my
consumer can't manage my partition well even when there are idle threads
then I have a bad design.
I can't design a system that in some situations doesn't deliver thousands
of emails because one thread couldn't manage things we
So, the general scalability approach with kafka is to add more partitions
to scale. If you are using consumer groups and the High Level Consumer
API, redistribution of partitions is automatic on a failover of a member
of a consumer group. But, the High level consumer doesn¹t allow a
configuratio
Hi Everybody,
I have 2 question regarding the way consumers, consume messages of a
partition.
- * Is it possible to configure Kafka to allow concurrent message
consumption from one partition concurrently? The order is not my concern at
all.*
I couldn't find any way to that b
