Re: How Producer handles Network Connectivity Issues

2015-05-29 Thread Bhavesh Mistry
Hi Kamal, In order to monitor each instance of producer, you will need to have alternative network monitoring channel (e.g Flume or Another Kafka Cluster for just monitoring a producers at large scale). Here is detail: 1) Add Custom Appender for Log4J and intercept all logs of Kafka Producer jav

Re: NoSuchElementException while retrieving logStartOffset

2015-05-29 Thread Stevo Slavić
Correction - restart did help, those exceptions are not logged anymore. Still, it seems there is a bug in topic deletion, some kafka.log.Log references continue to live after topic is deleted, until broker is restarted. Will try to create reproducible failing test. On Fri, May 29, 2015 at 11:50 PM

NoSuchElementException while retrieving logStartOffset

2015-05-29 Thread Stevo Slavić
Hello Kafka community, We had a ton of test topics, and deleted them using Kafka admin scripts - then our metrics error log started filling up with exceptions. Kafka metric reporter is trying to read LogStartOffset gauge value, and that throws NoSuchElementException. java.util.NoSuchElementExcept

Broker error: "failed due to Leader not local for partition"

2015-05-29 Thread Casey Sybrandy - US
Hello, For some reason, I'm getting this "failed due to Leader not local for partition" on my brokers. I can't send the logs, but I'm going to try to describe what I see. First, note that this is a fresh set of brokers with no data. I do see a series of messages stating "Expanding ISR for

Re: Kafka Not Commiting Messages

2015-05-29 Thread Charlie Mason
Hi All, Thanks for the help I have worked out what the issue was. The Kafka producer I guess was trying to connect to the producer using the machines hostname. I was using an IP Address in the server list. So it could connect into get the meta data. As soon as it then tried to connect to the leade

Re: KAFKA 0.9 release timeline

2015-05-29 Thread Joe Stein
Hey Murthy, we discussed 0.8.3 a couple months back https://mail-archives.apache.org/mod_mbox/kafka-dev/201503.mbox/%3ccafc58g8qpqy1mwd3w3oobexepdmgyy9gzuxcnovgesg1txd...@mail.gmail.com%3E we should be in a better place in the next few weeks for when that could ship. A lot of what you might have be

KAFKA 0.9 release timeline

2015-05-29 Thread Kuchibhotla, Murthy
Hi, I am checking to see when do we get KAFKA 0.9 version - where the basic authentication (security) is being implemented? Thanks -- Murthy Kuchibhotla The information contained in this e-mail, and any attachment, is confidential and is intended solely for the use of the intended recipient.

questions

2015-05-29 Thread ram kumar
hi, can i consume messages after a particular size is stored ?