Re: New Java Producer: Single Producer vs multiple Producers

2015-04-24 Thread Navneet Gupta (Tech - BLR)
Hi, I ran some tests on our cluster by sending message from multiple clients (machines). Each machine had about 40-100 threads per producer. I thought of trying out having multiple producers per clients with each producer receiving messages from say 10-15 threads. I actually did see an increase i

Getting best throughput without increasing latency with new producer/multiple threads producing

2015-04-08 Thread Navneet Gupta (Tech - BLR)
Hi, I wanted to understand what kind of config settings will help me get the best possible throughput with new producer API without increasing latency. I tried to run the new producer shared between couple of threads and latency seems to be increasing by increasing no. of threads. Ideally I would

Increasing replication factor of existing topics

2015-04-07 Thread Navneet Gupta (Tech - BLR)
Hi, I got a method to increase replication factor of topics here However, I was wondering if it's possible to do it by altering some nodes in zookeeper. Thoughts/suggestions welcome. -- Thanks & Regards, Navneet Gupta

Re: Alter command does a no-op for a topic instead of returning message about passing options

2015-04-06 Thread Navneet Gupta (Tech - BLR)
Ok. Looks like there is a JIRA task for that. https://issues.apache.org/jira/browse/KAFKA-1543 On Mon, Apr 6, 2015 at 1:30 PM, Navneet Gupta (Tech - BLR) < navneet.gu...@flipkart.com> wrote: > Sorry sent early by mistake. > > I tried this command > > /usr/share/fk-3p-kafk

Re: Alter command does a no-op for a topic instead of returning message about passing options

2015-04-06 Thread Navneet Gupta (Tech - BLR)
been given because I passed no options for what to alter in the topic. On Mon, Apr 6, 2015 at 1:27 PM, Navneet Gupta (Tech - BLR) < navneet.gu...@flipkart.com> wrote: > Hi, > > I tried this command > > > -- > Thanks & Regards, > Navneet Gupta > > -- Thanks & Regards, Navneet Gupta

Alter command does a no-op for a topic instead of returning message about passing options

2015-04-06 Thread Navneet Gupta (Tech - BLR)
Hi, I tried this command -- Thanks & Regards, Navneet Gupta

Re: using 0.8.2 in production

2015-03-30 Thread Navneet Gupta (Tech - BLR)
The latest available version is 0.8.2.1 Download it from here One of our teams have moved to this version in production and have reported no issues. On Mon, Mar 30, 2015 at 7:24 PM, pushkar priyadarshi < priyadarshi.push...@gmail.com> wrote: > Hi, > > I r

Re: kafka audit

2015-03-23 Thread Navneet Gupta (Tech - BLR)
Are there any plans to open source the same? What alternates do we have here? We are building an internal auditing framework for our entire big data pipeline. Kafka is one of the data sources we have (ingested data). On Mon, Mar 23, 2015 at 1:03 PM, tao xiao wrote: > Linkedin has an excellent t

GenericJMX plugin file for Kafka 0.8.1

2014-09-09 Thread Navneet Gupta (Tech - BLR)
Hi, We plan to capture various metrics exposed in Kafka via JMX and wanted to check if anybody has a plugin file which could be reused. We found one such plugin file here but it seems to be targe