[
https://issues.apache.org/jira/browse/KAFKA-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14002028#comment-14002028
]
Jim Hoagland commented on KAFKA-1448:
-------------------------------------
Having separate topics is the probably the best choice, but is not always
possible, e.g. if the way to split up a set of messages is not know ahead of
time or is dynamic.
> You can achieve filtering by writing a wrapper around your Kafka consumer
> that returns messages that pass the filter. There isn't much
> advantage to exposing this in Kafka APIs since there are no IO/CPU savings
> that can be achieved by doing that.
Not sure if this is worth adding to Kafka, but one advantage to filtering with
the Kafka APIs would be that the unneeded messages would need to traverse the
network, saving on network bandwidth.
> Filter-plugins for messages
> ---------------------------
>
> Key: KAFKA-1448
> URL: https://issues.apache.org/jira/browse/KAFKA-1448
> Project: Kafka
> Issue Type: Wish
> Components: consumer
> Reporter: Moritz Möller
> Assignee: Neha Narkhede
>
> Hi,
> we use Kafka to transmit different events that occur on different products,
> and would like to be able to subscribe only to a certain set of events for
> certain products.
> Using one topic for each event * product combination would yield around 2000
> topics, which seems to be not what kafka is designed for.
> What we would need is a way to add a consumer filter plugin to kafka (a
> simple class that can accept or reject a message) and to pass a parameter
> from the consumer to that filter class.
> Is there a better way to do this already, or if not, would you accept a patch
> upstream that adds such a mechanism?
> Thanks,
> Mo
--
This message was sent by Atlassian JIRA
(v6.2#6252)
