Re: Role of Producer

2016-07-13 Thread Snehal Nagmote
I think the system has too many producers. It is waste of > money. > > > Best, > Chao > > -Original Message- > From: Snehal Nagmote [mailto:nagmote.sne...@gmail.com] > Sent: Wednesday, July 13, 2016 3:38 PM > To: users@kafka.apache.org > Subject: Re: Role of

Re: Role of Producer

2016-07-13 Thread Snehal Nagmote
Hi Chao , To solve this problem , I can think of creating rest api . Your end point can have one of the parameter as data provider if you want to send it to different topics based on data provider . On backend , when you get data , you can send it to Kafka Topics, using Kafka Producer at the end

Re: Kafka Consumer for Real-Time Application?

2016-07-11 Thread Snehal Nagmote
Hello *,* If I understand your question correctly , what you are looking for is a setting in consumer which will only read latest messages . auto.offset.reset = latest This way , when you start new consumer group , it will always start reading from the recent offset, Thanks, Snehal On 1

Re: Question on partitions while consuming multiple topics

2016-07-05 Thread Snehal Nagmote
/main/java/org/apache/kafka/streams/processor/DefaultPartitionGrouper.java (Kafka Stream is supported with 0.10.0) Thanks, Snehal On 5 July 2016 at 11:47, Snehal Nagmote wrote: > Hello Yardena , > > You may want to take a look at manual assignment for partitions section > mentioned her

Re: Question on partitions while consuming multiple topics

2016-07-05 Thread Snehal Nagmote
Hello Yardena , You may want to take a look at manual assignment for partitions section mentioned here , http://www.confluent.io/blog/tutorial-getting-started-with-the-new-apache-kafka-0.9-consumer-client . However I have not tried using this for multiple topics , but looking at api , it should

Re: custom kafka reporters without JMX

2014-02-04 Thread Snehal Nagmote
I recently used this https://github.com/criteo/kafka-ganglia to do the same.It works well and pretty straightforward. I ran into some issues with dependencies of Scala version with Kafka (built with Scala version). For me it worked when I used Scala 2.8.2 with this plugin and Kafka with 2.8.0 Th