Re: Better strategy for sending a message to multiple topics

2015-12-20 Thread Abel .
Hi, Jens, I would do it that way but different queues contain different contents. That's why I need more than a single queue to solve my problem. However, I have a hierarchical structure of queues (like a tree or forest) and when you push a message to a queue I need to make it available to all the

Re: Better strategy for sending a message to multiple topics

2015-12-20 Thread Dong Lin
Did you try asynchronous produce? That should provide better performance because asynchronous produce doesn't block waiting for response. BTW, what is your ack setting for producer? Dong On Fri, Dec 18, 2015 at 1:16 PM, Abel . wrote: > Hi, > > I have this scenario where I need to send a messag

Logstash 1.5.3 with Kafka 0.9’s quota

2015-12-20 Thread Debraj Manna
In Kafka 0.9 support for quota is added as mentioned here . I am having some doubts about the behavior of quota in accordance with logstash. So in Kafka 0.9 quota can be configured for each client-id. If one of the client violates the q

Re: Monitoring MirrorMaker

2015-12-20 Thread Rajiv Jivan
Thanks for the pointers Pablo. Burrow looks like a really good fit. I'll update this list with my findings. On Friday, December 18, 2015 8:54 PM, Pablo Fischer wrote: We ran through this a few months ago, here is a list of things and tools I'd recommend you: - install burrow. It monitors the

Minimal KakfaConsumer in Scala fails compilation with `could not find implicit value for evidence parameter of type org.apache.flink.api.common.typeinfo.TypeInformation[String]`

2015-12-20 Thread Peter Vandenabeele
Hi, I am trying to write a minimal Kafka consumer in Scala and got this far: ➜ scala git:(kafka_exp_001) ✗ cat KafkaConsumer.scala package io.allthingsdata.kafkaConsumer import org.apache.flink.api.scala._ import org.apache.flink.streaming.api.scala._ import org.apache.flink.streaming.util.seri