Hi Using Storm would be another way. This will scale as well. Spark streaming would fit as well
It all depends on the complexity of the filter and any additional processing required. HTH Lohith Sent from my Sony Xperia™ smartphone ---- Gerard Klijs wrote ---- Using kafka streams is one way, I used camel before with kafka, which also has a nice way of using filters. On Fri, Apr 29, 2016 at 1:51 PM Subramanian Karunanithi <subub...@gmail.com> wrote: > Hi, > > When a stream of data passes through Kafka, wanted to apply the filter and > then let that message pass through to partitions. > > Regards, > Subramanian. K > On Apr 26, 2016 12:33, "Marko Bonaći" <marko.bon...@sematext.com> wrote: > > > Instantly reminded me of Streams API, where you can use Java8 streams > > semantics (filter being one of them) to do the first thing in Gouzhang's > > response (filter messages from one topic into another - I assume that's > > what you were looking for). > > > > Marko Bonaći > > Monitoring | Alerting | Anomaly Detection | Centralized Log Management > > Solr & Elasticsearch Support > > Sematext <http://sematext.com/> | Contact > > <http://sematext.com/about/contact.html> > > > > On Tue, Apr 26, 2016 at 6:22 PM, Guozhang Wang <wangg...@gmail.com> > wrote: > > > > > Hi Subramanian, > > > > > > Could you elaborate a bit more on "filtering"? Do you want to read raw > > data > > > from Kafka, and send the filtered data back to Kafka as a separate > topic, > > > or do you want to read raw data from an external service and send the > > > filtered data into Kafka? > > > > > > > > > Guozhang > > > > > > > > > On Tue, Apr 26, 2016 at 8:07 AM, Subramanian Karunanithi < > > > subub...@gmail.com > > > > wrote: > > > > > > > Hi, > > > > > > > > Do we have any plugin which is available, which can be used as a > > > filtering > > > > mechanism on the data it's working on? > > > > > > > > Regards, > > > > Subramanian. K > > > > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > > Information transmitted by this e-mail is proprietary to Mphasis, its associated companies and/ or its customers and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly prohibited. In such cases, please notify us immediately at mailmas...@mphasis.com and delete this mail from your records.
