Re: Ephemeral ports for Kafka broker

Thanks, Ewen. I did know passing 0 would work, but didn't find an API to get back the bound port. Which version of Kafka library is that in? We are using 0.8.2.0 Kafka and it doesn't seem to be available there? Thanks Hemanth On Wed, Nov 11, 2015 at 11:39 PM, Ewen Cheslack-Postava wrote: > Pass

Can I use JMX reporter for getting the metrics into zabbix?

Hello All, We have multiple metrics reporters for publishing/viewing the kafka metrics. I want to dump all kafka metrics in *zabbix.* I am thinking the following approaches to get the metrics in to zabbix. 1) Use JMX reporter in kafka to get the metrics and accessing these metrics from zab

Re: JMX prot

Take a look here: https://kafka.apache.org/08/quickstart.html Basically, you need to set the JMX_PORT variable before starting Kafka From: , Roy Sent: November 11, 2015 3:19 PM To: users@kafka.apache.org Subject: JMX prot Hi We are using kafka 8.2.1 w

JMX prot

Hi We are using kafka 8.2.1 want to know if we need to configure jmx port in kafka config ? if no then how do I tell kafka to expose JMX metric on specific port ? any idea how to do this ? Thanks roy

Re: Blocked consumer - next offset same as current offset

Thanks Jun for heads up! Looked it up in wiki page: https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example#id-0.8.0SimpleConsumerExample-ReadingtheData "Also note that we are explicitly checking that the offset being read is not less than the offset that we requested. This

Re: Ephemeral ports for Kafka broker

Passing 0 as the port should let you do this. This is how we get the tests to work without assuming a specific port is available. The KafkaServer.boundPort(SecurityProtocol) method can be used to get the port that was bound. -Ewen On Tue, Nov 10, 2015 at 11:23 PM, Hemanth Yamijala wrote: > Hell

Re: Blocked consumer - next offset same as current offset

Are you using compressed messages? If so, when using SimpleConsumer, it's possible for you to see messages whose offset is smaller than the offset in the fetch request, if those messages are in the same compressed batch. It's the responsibility of the client to skip over those messages. Note that t

Re: Blocked consumer - next offset same as current offset

Not sure if related, but this line in AbstractFetcherThread (Kafka 0.8.2 branch) looks suspicious https://github.com/apache/kafka/blob/0.8.2/core/src/main/scala/kafka/server/AbstractFetcherThread.scala#L122 This logic seems to have been changed significantly via https://github.com/apache/kafka/com

Why do we need broker id in the Zookeeper directory for consumer offset?

According to the documentation: Consumer Offsets Consumers track the maximum offset they have consumed in each partition. This value is stored in a ZooKeeper directory if offsets.storage=zookeeper. This valued is stored in a ZooKeeper directory. /consumers/[group_id]/offsets/[topic]/[brok

Blocked consumer - next offset same as current offset

Hello Apache Kafka community, I'm using simple consumer with Kafka 0.8.2.2 and noticed that under some conditions fetch response message set for a partition can contain at least one (if not all) MessageAndOffset with nextOffset being equal to current (committed) offset, offset used in fetch reque