Raj created KAFKA-10280:
---------------------------
Summary: Message filtering support based on keys format/Headers
Key: KAFKA-10280
URL: https://issues.apache.org/jira/browse/KAFKA-10280
Project: Kafka
Issue Type: Improvement
Components: consumer, core
Reporter: Raj
There are many scenarios where there is a need for consumers to subscribe to a
message based on pattern eg. Key format or headers.
Typically, this scenario gets solved using external components implementing a
filtering logic.
In a deployment with a large number of consumers,This becomes a huger
performance (network/IO) overheads if most messages are getting discarded by
the consumer based on the filter. If there are 10000's of consumers subscribing
to the topic partitions, there is unnecessary IO, which could be avoided broker
can apply filter per subscriptions.
I wanted to hear insights from the Kafka community around how they are solving
this problem and gauge interest in formally submitting KIP around filtering as
part of core capability.
Thanks,
Raj
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
