Thanks everyone, shall try these options.
Regards,
Subramanian. K
On Mon, May 2, 2016 at 9:43 AM, Andrew Otto wrote:
> If you want something really simple and hacky, you could use kafkatee[1]
> and kafkacat[2] together:
>
> kafkatee.conf:
>
> input [encoding=string] pipe tail -f a.log
> output
If you want something really simple and hacky, you could use kafkatee[1]
and kafkacat[2] together:
kafkatee.conf:
input [encoding=string] pipe tail -f a.log
output pipe 1 grep -v ’not this’ | kafkacat -P -b b1:9092 -t mytopic
[1] https://github.com/wikimedia/analytics-kafkatee
[2] https://github
Apache Samza is the way to go. Never used Kafka Streams so no opinion on that
one.
Best regards,
Radek Gruchalski
ra...@gruchalski.com (mailto:ra...@gruchalski.com)
(mailto:ra...@gruchalski.com)
de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/)
Confidentia
Hi
Using Storm would be another way. This will scale as well.
Spark streaming would fit as well
It all depends on the complexity of the filter and any additional processing
required.
HTH
Lohith
Sent from my Sony Xperia™ smartphone
Gerard Klijs wrote
Using kafka streams is one way
Using kafka streams is one way, I used camel before with kafka, which also
has a nice way of using filters.
On Fri, Apr 29, 2016 at 1:51 PM Subramanian Karunanithi
wrote:
> Hi,
>
> When a stream of data passes through Kafka, wanted to apply the filter and
> then let that message pass through to
Hi,
When a stream of data passes through Kafka, wanted to apply the filter and
then let that message pass through to partitions.
Regards,
Subramanian. K
On Apr 26, 2016 12:33, "Marko Bonaći" wrote:
> Instantly reminded me of Streams API, where you can use Java8 streams
> semantics (filter being
Instantly reminded me of Streams API, where you can use Java8 streams
semantics (filter being one of them) to do the first thing in Gouzhang's
response (filter messages from one topic into another - I assume that's
what you were looking for).
Marko Bonaći
Monitoring | Alerting | Anomaly Detection
Hi Subramanian,
Could you elaborate a bit more on "filtering"? Do you want to read raw data
from Kafka, and send the filtered data back to Kafka as a separate topic,
or do you want to read raw data from an external service and send the
filtered data into Kafka?
Guozhang
On Tue, Apr 26, 2016 at
